Changing a Test Case Management Tool into a Well Running Project
Insights and Tips Based on a True Story
Latest posts by Sergiu Tarnovan
There may be many factors that lead to a major change, a change that affects many people involved in the software development process, but I’d like to talk a little bit about changing a test case management tool and the impact that it can have on a project that works very well from all points of view, from people to processes and well-meaning business.
I read a lot of articles about “What is the best test case management tool?” and “Why do we choose X, Y test case management tool?” and all sorts of ratings and charts, but I have not seen a description of the process of changing a test case management tool for a running project, so I decided to gather some insights on this.
What is a test case management tool and why do we need it in the software development process?
Test management tools are the ones that are used to keep and maintain a record of the way in which testing is to be done, plan its activities and update the status of various quality assurance activities.
Why do we need to change a test case management tool?
There can be a lot of factors that can influence a company or a project to switch from one test case management tool to another. I’ve gathered and listed below a few of them:
One of the most important factors is the cost of the test case management tool. The cost of a solution should be looked into while selecting a new test management tool. This involves taking into account not only the upfront amount it costs to buy a tool, but also training, implementation and support costs. Pricing can be a blocker when selecting the test case management tool, so this needs to be carefully analyzed and adapted to the project or company budget.
- Legal Constraints
Legal concerns may occur, which would weigh heavily in the list of cons against picking a certain test case management tool. Here are a few examples:
- Some terms and conditions may enable the software provider to read or even publish the data contained within the test cases. This poses not only a legal risk, but also a security risk as well, given that the data which should only be accessible by members of your company may now be accessed by unlicensed third parties.
- They may use your company logo without prior authorization to advertise that you are their client and use their software, be it either on their website or social media.
- Let’s say your company is based in X country and the test case management tool provider is based in Y country. Some Terms and Conditions might be governed under the laws of the country where your provider is based (Y), whereas your company may require that all agreements must be governed under the laws of the country that your company is based in (X).
Though you may have these legal concerns, don’t scrap the tool yet. Instead, try to reach a consensus in which both parties – your company and the provider’s company – are in agreement, since it may be at your disadvantage to discard an otherwise useful tool due to some technicalities (this is, of course, depending on the scale of said technicality).
- Features and Functionalities
The lack of basic functionality of a test case management tool can lead to change. Among the most important features are the ones listed below and the lack of them is an important factor in the decision to change a tool:
- Flexibility and ease of use
It is important for any test management tool to be easy to use and flexible. Ensuring this helps testers in identifying any sort of errors that might be affecting the functionality of the product.
- Streamlined tracking capabilities
In order to ensure that a project progresses smoothly from one stage to another, it is important to keep a check on each step of the testing life cycle. Maintaining a record of different stages of testing in the form of testing dashboard benefits not only the current project but also the future as it allows one to easily detect any sort of incompatible testing requirements.
- Real-time reporting
Maintaining the complete documentation based on real-time testing of the product allows for easy tracking. Reporting in real-time also helps in analyzing what’s happening at the present moment in a particular project. This also allows one to get quick and instant responses from the QA teams and also enable its flexibility.
- Integration of automation testing
Automation testing has, undoubtedly, become an integral part of the entire software development process. Allowing this feature in software not only provides ease to the testers but also saves a lot of their time. Whether it is conducted as per the predefined cycle or as needed, automatically running the tests implies that the team won’t have to remain occupied with this work.
- Easy tracking
The test management tool should essentially be able to assign a particular task to a particular individual. It is also important to ensure that it is possible to track whether the action assigned has actually been performed, when and by whom.
- Ensured security
Test management tools should also offer complete security to the test assets as well as all the relevant and important information about the test projects. With high-level security, it is possible to distribute test data with ease and without stressing about its unauthorized access.
When there is a team working on a particular project, it is important to ensure that its access is available to all via cloud servers. This helps in creating an environment that allows for an easy exchange of information about the project among different team members.
How tough is it to change a test case management tool into a well-running project?
Once you’ve figured out the reasons for why a test case management tool requires change, the impact of this change and the actual change process needs to be analyzed.
The decision to change can be difficult at first, but with a well-established process, it can be an easy and smooth transition and come with benefits for the project or company.
Going forward, I have gathered some details that underline the main aspects of changing a test case management tool and divided them into three main phases:
Phase 1 – Initial requirements identification
Each project may have different requirements given the many variables within said project. Test case management tools, while at first might seem like a simple utility, they are a great example of the plethora of requirements within a certain project. For the sake of readability, let’s split them into a few groups.
- APIs, plugins and integrations
- In the event of migrating from another test case management tool, we’re going to need to have the ability to bulk import or create test cases and test suites, since rewriting them will take a long time
- Automation runs will need to be able to publish the test results in the test case management tool via an API, in order to evaluate their release quality
- JIRA/Custom chats/CI integration would be great for ease of creating bug tickets or publishing them via a custom chat channel
- E-mail integration for publishing reports and test result notifications
- User management. It goes without saying that not every user needs to have admin level permissions or the ability to accidentally delete tests/test runs
- SSO is pretty much everywhere right now and most IT companies use some type of secure logins such as Onelogin, ADFS, Okta, etc.
- Test suite/test case management
- Testers should have the ability to create/edit/delete/sort test cases, organize them into test suites, sections and sub-suites
- One of the most basic requirements is the ability to create and execute test runs and mark the tests as pass or fail. While pass/fail is the minimum requirement, we might also need to have other statuses such as N/A, Blocked or Retest
- The ability to edit and copy test cases from/to other test suites for better separation. For example, in the event where we would need to separate the tests by platform. The tests may be similar, but still have their own particularities in terms of platform that it is executed on.
- When writing new tests, ideally one would be able to write separate steps and expected results so that it would make it easier to mark an individual step as pass or fail
- Triggering automation runs from within the test case management tool would also be a major plus, so testers wouldn’t need to jump through a multitude of other tools to do so. This is especially helpful in the case where you would need to schedule automated test runs (i.e.: for time sensitive test cases).
These requirements should be shared with the test teams and stakeholders in order to draw a bigger picture of what the teams need to run their day-to-day. Stakeholders will (or at least should) adjust their options based on the testers’ feedback.
Phase 2 – Identify and Evaluate Alternatives
Once the initial requirements identification part is complete, it’s time to research and analyze any and all alternatives that you can come up with and compile a list of all the available products.
When a shortlist of alternatives is in place, you can start trialing them and communicate the trial to others and figure out which solution works best given the list of requirements and any other concerns.
The trialing process is as important as the requirements identification from Phase 1 and is a determining factor in choosing the new test case management tool. Here we have gathered documentation with instructions based on whether we determine if the new test case management tool covers enough things to be able to evaluate against the requirements and the priorities set during that phase.
After finally settling on the tool that best matches your use case, it’s time to set up the legal and procurement sign off procedure.
Phase 3 – Migrate to Chosen Test Management Tool
In order to start the migration to the chosen test case management tool, you need to determine a plan and see what the best approach is. This plan should be based on previous phases and the feedback gathered.
Once the migration plan is all figured out, the technical part should be fairly straightforward. You’d have to export the test cases, attachment and any other odds and ends from the previous test management software to the new one.
Migrating such a large dataset might pose quite the problem, but the vast majority of test case management software support exporting test cases or even test suites to easily manageable formats such as .CSV/.XML or even Excel. These formats enable the best possible way to keep the test case structure format intact. If your new tool only supports one of these formats – say it doesn’t support CSV, but does support XML- it should be fairly easy to convert between them and import.
The migration procedure will vary based on the platforms you’re working with, but if you’ve done your homework right, it should be a breeze.
All that’s left after this step would be to set up the user accounts/permissions, provide documentation and ramp up your teams on how this new test management tool works. Then prepare to wave goodbye and cutover to the new platform.
At the end of changing a test case management tool, we can analyze and compare if the new tool meets the team needs in the day-to-day tasks and if it comes with the desired requirements and improvements that were agreed upon during the changing decision. All these aspects should clearly underline whether the change was beneficial or not.
There are certain moments in a project lifecycle where, constrained by one or more factors, we need to make tough decisions, like the one of changing a test case management tool. Why and how we make this change may differ from case to case, but most of the transitions have common points as those mentioned above and it would probably be a good starting point for teams and projects that will encounter this kind of challenge.