“Thank you so much for handling this.” – This is how she approached me after we pushed the fix out to Production.
“Sure thing. It was a team effort.” – I replied proudly.
“We need to address this in the Retrospective meeting.”
“I agree, and I would like to share some of my ideas on how could we prevent this in the future.”
Not so long ago I was reading about an online seminar, Software Testing Trends, where discussions about two new skills/techniques got my attention. The reason was that they didn’t promise a bug-free release, but instead assured they would give us the power of knowledge: Knowing why the bugs were missed in the first place, and how to minimize their appearance and impact.
Another interesting fact about these skills was that they are focused on being in line with our already in place processes. This helped us manufacture information faster, sooner and more efficiently, which ultimately is our job.
Imagine yourself in 5th grade with a huge white piece of paper and a red marker. You write your name in the center of it, and you draw a circle around it. You also draw multiple arrows running out of the circle, each of them reaching a rectangle with some categories in it: Family, friends, hobbies, pets, etc. Do you see it?
As you remembered a similar situation from your childhood, you instantly visualized almost exactly how you drew that personality map. Do you remember the content of any of the essays you wrote in the same school year? Unless you’re the proud owner of a superpower like “Enhanced Memory“, you probably won’t.
This is mind mapping; it helps visualize the thinking process.
How can we use mind maps for work?
- Test planning – much more efficient than a 100-page test plan (which is never fully read by anyone), it’s not flexible when it comes to updates/changes, and it also goes out of scope quickly. With mind maps, we can make a distinction between the main areas we want to focus on: Tasks, time allotted, tools, risks, details about the environment, etc. The final map will show us at a quick glance the whole scope of the plan.
- Test cases – we can create branches on our map for every user story/epic/feature, and we can start adding ideas/tests for each function we have available. The same report can be used for test execution purposes, offering an overview of the passed/failed tests, issues found, links to attachments, etc. The same one can also be used for test reporting purposes. So our mind map becomes dynamic documentation about the testing cycle. We should be getting better coverage out of this.
- Project dashboard – progress, schedule, issues, risks. We can rely on the mind maps to see the bigger picture and understand the status and health of the project.
- Meeting Minutes – agenda, attendees, time and place, discussions, action items, docs, notes, etc. – Everything in one place.
- Analysis – Mind maps are a neat way to remember and retain information that we read! I personally was amazed at how fast I started to understand the information when I mind map. A quick scan through the mind map enables me to restore the information back into my mind.
As we all know, traditional test artifacts are time-consuming, their structure is anything but compatible with the agile approach of the projects we are are working on in 2017.
The information overload with which we are coping daily reduces our efficiency and effectiveness. This is how issues like lack of creativity, forgetfulness, lack of focus and clarity, and other similar behaviors are coming to surface. Using mind maps as often as possible helps us keep everything organized, in the same place, and the best part is that in time and with practice it can be the root cause of creative thinking. We can develop ideas while drawing branches.
We are asked to create countless documentation during the testing process. The time we spend working on these artifacts is the time that could be spent actually testing. A mind map can hold the same amount of information in fewer words, and is easy to create, to modify, to maintain and to review.
Free mind mapping software
Many free mind mapping tools are available online. You can try any mind map tool which works for your ideas. Here are a few that I already tried out:
Also, here is a list of the top 10 mind mapping software programs from Digital Trends.
Pass vs. Fail ratios
Some might say that the client needs numbers to have an actual way of measuring quality. Let’s see why this might be problematic.
Calculating a pass and fail ratio is a relatively easy task, but if it’s used to estimate the state of a project, the ratio will be invalid, irrelevant and could even be misleading.
- Just because a test case is marked as “passed” does not guarantee that the product will work correctly and that it’s reliable. It only suggests that a test is according to the expected results, under a specific set of conditions, under a particular environment.
- Or if a test suite is marked as entirely passed on a specific environment – Which test cases have been covered, under what conditions, what were the priorities? All of these are variables and changing only one could result in a failed test/test suite.
So a passing test is a rumor of success.
- At the same time, a failed test case does not prove that our product is not working. The failure could be a cause of a misunderstanding when it comes to the requirements, the test running procedures, some configuration mistakes, wrong or insufficient test data, etc. Basically, it can be a not yet understood reason for why the product is producing a result. A result which seems incorrect, but which could be correct. So a failing test is an allegation of failure.
Ok. So what is the alternative?
Jerry Weinberg came up with a definition for “measurement” which, in my opinion, a lot of testing specialists more in line with what the client would want: “Measurement is the art and science of making reliable (and significant) observations.” This definition incorporates the meaning of qualitative measurement
As software testers, we are asked to deliver a lot of immediate and qualitative information. There is a need for more than just a number, so we have to learn to describe and report about our product in a test, the testing that we are running and the quality of our testing. This includes creating, editing, narrating and justifying a story.
Delivering the news
We can see this method of delivering results as delivering the daily news. Testing is like investigative journalism. We need to research and provide a story to a group of people. A good journalist knows how to get our attention to exactly what interests us efficiently, so we can find the information or detail that we require.
In testing, usually, the potential showstoppers are the most important stories. Taking this into consideration, on the first page we need to print the issues with the most impact, and here, usually, we can highlight and illustrate its impact on the business.
How do we write our story? Which stories will receive the front page spot in our newspaper?
- Bugs with the biggest impact and major bugs will always be on the front page
- Non-problems (minor bugs, questions, etc.) aren’t news because they aren’t an immediate threat to the project
- A qualitative test report won’t be one that hasn’t any numbers. We should still talk about coverage, but we can do it differently:
- Level 0 means: “We know nothing about this area of the product.”
- Level 1: “We have done smoke or sanity testing. At this point, we’ve determined whether the product is even stable enough for serious testing.”
- Level 2: “We’ve tested the common, the core, the critical, the happy path. Our testing has been focused on ‘can it work?’”
- Level 3: “We’ve tested the harsh, the complex, the challenging, the extreme, the exceptional. If there were a serious problem in this area, we’d probably know about it by now.”
Storytelling in test reporting can be done on three levels. Let’s take a new feature as an example:
Level 1: Tell the product story
- How is the feature supposed to work?
- How does it fail?
- How could it fail?
- Report about the nature of the feature, targeted users, the presence or absence of the value for the targeted users and obviously the risks.
Level 2: To make the product story credible, tell the testing story
- Report about how we configured, observed and evaluated the feature
- Reveal what we actually did and what we noticed as a result of our actions
- Answer the following:
- Where we tested
- How we tested
- What did we find?
- Where didn’t we test?
- Where are we planning to test some more?
- Most likely our feature will have some bugs
- How did we find the bugs?
- Priority and severity of the bugs?
- We don’t conclude the state of the product at this point
Level 3: To make the testing story credible, tell a story about the quality of the testing
- Why the testing we performed and the way we performed it should be enough
- Why the testing we didn’t perform (yet) should not be a priority right now
- What made the testing easy or difficult?
- What risks should we expect at this point of the testing?
- What are the costs of the risks?
- What do we need in order to report on our testing faster and more accurately?
A few months passed since I first discovered these techniques and started to implement bits of them. I noticed a more focused, creative and organized person in myself. Even if the team and project that I currently work on haven’t suffered huge changes in regards of the process, my own process did, and that change positively impacted the work with which I contributed to the team.