Modernizing the Testing Day Experience

Game of Pods is an Agile Acceleration Platform designed to support the cultural side of digital transformation. In a world where 74% of enterprises report an inability to keep key engineering talent beyond one year and 1 in 5 digital projects fail… we’ve achieved the top talent retention rate in our industry and consistent success across digital product engineering through our own purpose built platform that focuses on metrics that matter.

A non-profit organization representing America’s most popular college entrance exam—one that is accepted and valued by all universities and colleges in the U.S.—had a lofty goal: “Modernize the Test Day Experience.” 

The organization called on Cognizant Softvision to build custom digital tools and platforms to improve the testing experience for all key stakeholders, including students taking the exam, as well as administrators, coordinators and employees who support test days across the U.S. from the “Command Center” in Iowa City, Iowa.

When the engagement began, the Cognizant Softvision team walked the client through the process to define differentiated digital experiences and tools for test administrators, students and the organization’s Command Center. We determined that several new software and technology solutions were needed, especially in light of COVID-19, which would drastically impact how the client-initiated tests. Enabling these applications required modernizing back-end systems and drawing a clear roadmap for how to deliver these experiences and key features. 

We curated two remote, dedicated teams, called “Virtual Pods,” to develop the client’s software and technology, starting with web and mobile applications for the test administrators and coordinators. 

Zero Gravity, one of the pods, composed of 10 experts in Design & UX, Enterprise Software, Quality Engineering, Full Stack Web and PMO, is dedicated to helping accelerate the delivery of a new application that will target the state and district business side of testing as opposed to just school testing. This application will be a premium B2B product that is targeted to launch early next year. While our initial app covers the national testing experience and is recognized as an anchor of consistent successful releases/test events, with roughly 2,000 locations and over 100,000 students on test day taking the exam, the application helps facilitate the influx of tests, as custom applications enable administrators to undertake the pre-test planning process, manage the test day and complete the post-test set of processes, all digitally.

A Challenge Appears! Track, Measure, Release, Repeat 

Our pods hold themselves to distinct KPIs within Game of Pods that measure velocity, quality, product impact, and autonomy to ensure that our delivery is seamless and a product of excellence – built for outcomes. 

As Zero Gravity was preparing for the Alpha release, the team noticed during a retrospective session that the number of milestones they committed to the client versus the ones they completed, otherwise known as the Say/Do ratio, fluctuated. Said another way, the team was suddenly — for a number of reasons — not hitting their goals. 

The Say/Do ratio is a velocity KPI, measuring the amount of work a team completes during a sprint, by forecasting. The product owner uses this velocity KPI to predict how quickly a team can work through features. 

Although the team was tracking the progress, it wasn’t listed as a primary velocity metric within Game of Pods. Since this was a vital feature that needed to be released on time, the team adjusted their focus to adding the Say/Do ratio to their Game of Pods track and increasing the Say/Do ratio from an average of 80% to 100% over the next four sprints. 

In short, they used Game of Pods to put Say/Do in focus and to re-commit themselves, as a team, to delivering on 100% on time and as promised. 

Why the problem was happening

A fluctuating Say/Do ratio is a sign that a part of the team’s development process has become inefficient and should be brought up at the next retrospective. 

With this new metric put in center-focus within GoP,  it was discovered that our client, having internal resource challenges of their own, had turned to our team of experts to help with production of one of their internal applications, which ultimately left little time to make progress on the committed project. 

“Our team, notably a group of determined and passionate colleagues, took on as many responsibilities as we could to help our client with their challenges they were facing, while ensuring we didn’t fall behind. As the release cycle crept up, it became clearer that while we were supporting our client with other high-pressing issues, we needed to keep moving forward on our committed tasks,” says Ovidiu Crisan, project manager and scrum master. “Knowing that this was going to be our first release cycle and highly anticipated, we wanted to make sure that we were pushing our best selves across the board and adding the new metric to Game of Pods was that driving factor.”

On average, the Say/Do ratio before Game of Pods was 83% per sprint. Game of Pods optimized the efficiency of the pod, highlighting where they should be focusing their priorities, and ensuring consistency. Once the metric began to get officially tracked it became clear, sprint after sprint, that Game of Pods shifted focus for the pod, helping to project progress upward – stabilizing the Say/Do ratio to above 98% for the last six sprints before release. On the sixth sprint, Zero Gravity increased to a 100% ratio and received points and recognition within the Game of Pods platform, ultimately leveling up the pod. 

More than a Game

“Sure, the accolades and recognition are great motivating factors for us as a pod. It feels good to get recognition and be motivated by your success, but it was also the ability to showcase our efforts right there, on the track, to our client, so they could see first-hand our capabilities to deliver on commitments, while simultaneously assisting them with other out-of-scope challenges, “ Ovidiu explained. “Game of Pods very clearly showed us when we needed to get back on track and move away from assisting to focusing on things we were behind on.”

For our client, tracking the Say/Do Ratio gave insight into progress on the project – a rare opportunity not afforded with other companies. When asked about the results on the project, the client Digital Vice President stated, “Those are absolutely stellar results! I really appreciate all the great focus and leadership that the team shows. Can’t get better that 100% on that say/do. I understand why the committed numbers show the way they do, but it will be something to watch once [our company] gets back on track with our internal resource pool.” 

Being a part of a company committed to agility and being part of an agile pod, Zero Gravity is expected to deliver at speed in learning and quickly delivering, which means they are constantly measuring improvement and failures on regular frequencies helping organizations and teams stay on track with their commitments.  

With Game of Pods, the team realized their full potential – utilizing the codification platform to understand what greatness can be achieved, help their team understand the gaps and focus on metrics that matter throughout the product engineering process. For Zero Gravity, it was about solving a problem that mattered to their specific pod and helping them push for excellence.