Saving Time and Improving Quality Through Parallel Testing

Ionut

Ionut

Line Manager
Ionut started in Softvision back in 2007 and over the years he dedicated his career to Quality Assurance.Starting with 2014 he switched to a management role and he is currently responsible for more than 10 projects involving around 80 engineers.Along the way Ionut was a bold advocate for quality, improvement and innovation, always in search of new ways to infuse more of these values into his projects and teams.
Ionut

Latest posts by Ionut

Let’s face it, in the IT industry time is never your friend. From time to time you end up in a situation where the schedule is too tight and you need to be creative if you want to keep up with the deadlines and still deliver quality.

At one point when we were working with our client to build a key piece of software, and the schedule was extremely tight, we needed to assess the quality and overall usability of the application. We decided against involving the current team as we wanted an objective third party perspective. As we didn’t have very much time at our disposal, we came up with a strategy wherein a beta testing session and a usability testing session ran in parallel.

Beta Testing is also known as field testing. The goal of beta testing is to place your application in the hands of real users outside of your own engineering team to discover any flaws or issues from the user’s perspective that you would not want to have in your final version.

Advantages of beta testing:

  • You have the opportunity to get your application into the hands of users prior to releasing it to the general public.
  • Users can install, test your application, and send feedback to you during this beta testing period.
  • Your beta testers can discover issues with your application that you may have not noticed, such as confusing application flow or even crashes.
  • Using the feedback you get from these users, you can fix problems before releasing to the general public.
  • The more issues you fix that solve real user problems, the higher the quality of your application.
  • Having a higher-quality application increases customer satisfaction.
  • These early adopters will generate excitement about your application.

Usability testing. The goal here is to assess the ease with which the user interfaces can be used and whether the application or the product built is user-friendly.

Advantages of usability testing:

  • Usability test can be modified to cover many other types of testing such as functional testing, system integration testing, unit testing, smoke testing etc.
  • Usability testing can be very economical if planned properly, yet highly effective and beneficial.
  • If proper resources (experienced and creative testers) are used, usability test can help in fixing all the problems that user may face even before the system is finally released to the user. This may result in better performance and a standard system.
  • Usability testing can help in discovering potential bugs and potholes in the system which generally are not visible to developers and even escape the other type of testing.

As you see there are many things to be considered if you want to fully benefit from a beta and usability testing session. In our case the key elements for this parallel testing were: number of people involved, type of people involved (demographics, IT skills) and timing. Here is how we did it.

Organizing the troops:

  • With the help of our client we gathered 300 people to help us with the testing activities:
    • The first group having around 200 non-technical people, covered multiple demographic areas starting with college students and ending with grandparents, all located in US.
    • The second group was composed by 100 technical engineers and it was distributed between US and Romania
  • Starting from these two big groups we created 10 smaller groups, each one having 30 people and a designated owner.
  • Each owner was responsible to help his own group with answers, guidelines or by gathering the feedback at the end of the testing session.
  • We wanted to avoid any disruption for our teams so we discussed with everyone involved to avoid asking questions or discuss with peers as long as the session is active. All questions were addressed to each group owner.
  • We printed fliers with general guidelines available in each room where the testing teams were present. The information was available also online.
  • We held multiple presentations informing the testing teams about the application under test, goal of this testing and other general guidelines (e.g. How to access the application, credentials, what devices are allowed etc.)
  • We allocated one hour for the entire testing session.

Defining the testing scope:

  • The scope was functional and usability testing from an end user perspective so we drew a clear line in terms of what activities or tools are allowed or not during that testing session (e.g. security, performance or stress testing were blacklisted)

Gathering the results

  • We provided two options for submitting a bug report or general feedback: the classical pen and paper or a web form.
  • We asked every participant to do the following for any reported issue/Bug/ECR:
    • Reproduce the issue at least two times in order to be able to define clear steps to repeat it
    • Capture any important details that could help the development team (e.g. OS version, browser type, device type)
    • When possible attach any images or logs or any other resource that could help.
  • At the end of the testing session each owner gathered all feedback from their group.

Conclusion

First of all this testing experience was extremely cool! 300 people, two continents, one hour testing in sync, great fun! It was also the most productive testing hour we have ever spent. After analyzing all user reports we got the following numbers: 270 reported issues; 50 Possible Enhancements; 50 Not an Issue.
This was truly an amazing experience involving two great teams, partners and an awesome project!

Share This Article


Ionut

Ionut

Line Manager
Ionut started in Softvision back in 2007 and over the years he dedicated his career to Quality Assurance.Starting with 2014 he switched to a management role and he is currently responsible for more than 10 projects involving around 80 engineers.Along the way Ionut was a bold advocate for quality, improvement and innovation, always in search of new ways to infuse more of these values into his projects and teams.
Ionut

Latest posts by Ionut

1Comment
  • Anil Raj
    Posted at 11:31h, 18 July Reply

    Thanks for sharing your experience (or more like an adventure) of Parallel Testing. The idea of Beta testing with potential Users and gathering their feedback in order to improve the quality of the product is remarkable. This way a lot of scenarios can be covered from end user’s standpoint.

Post A Comment