BOOK CLUB: How to Reduce the Cost of Software Testing (13/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry.

We are now into Section 3, which is sub-titled "How Do We Do It?". As you might guess, the book's topic mix makes a change yet again. We have defined the problem. We've discussed what we can do about it. Now let's get into the nuts and bolts of things we can do, here and now. This part covers Chapter 12.

Chapter 12: A Nimble Test Plan; Removing the Cost of Overplanning by David Gilbert

As I posted in another blog post a couple of days ago, there is no more frustrating experience than having to put together a way too specific test plan, one that is under most circumstances never going to be completely run, and even in the best circumstances, doesn't even really provide the appropriate coverage needed (but it sure looks impressive and feels ever so hefty as it weighs in at several ounces if not closer to pounds).

These test plans are not just cumbersome and unwieldy, but they are also shackles that lock us in place and become more inefficient the longer the project progresses. Any later discovery and re-calibration is often discouraged in favor of covering the tests already defined, regardless of whether or not they have any real effect. By creating a "nimble" test plan, we focus on the test cases that are the most relevant, and we adapt it over time as the conditions determine.

Plain and simple a "plan" is what you believe you will do, at the time you devise the plan. Things change over the course of time. Being able to adapt and change is much more effective than a plan that was created earlier and locks the tester into a set approach. Much like a map that's never been made, a plan is only the best ideas an estimates of what may be happening. You won't know for sure until you actually try to follow it, and hey, your assumptions may prove to be wrong!

A nimble test plan will include direct communication and feedback, and it will be a living document that changes as new information comes to light. Nimble is defined as:

• quick and light in movement; moving with ease; agile; active; rapid: nimble feet
• quick to understand, think, devise, etc.: a nimble mind.
• cleverly contrived: a story with a nimble plot.

Nimble testplans are small, lightweight, adaptable to change and easy to read and understand. it is allowed to change and evolve as time progresses based on customer’s needs. They uses good ideas from many different disciplines.

To creating a nimble test plan, it's important to keep some key criteria in mind:

1. Keep It Light

The test plan should provide proper guidance for the scope of testing, but should not dictate the act of testing. Keep it Simple. Anything not adding value to the customer should be considered waste.

2. Make All Content Goal Oriented

Useful heuristics can be a big aid in this process, such as San Francisco Depot (SFDPOT): Structure, Function, Data, Platform, Operations and Time. Covering just these areas in a system and actually executing a testplan with just this heuristic can provide a huge advantage to testers over those who do not structure their approach along a similar heuristic.

3. Commit to as Little as Possible

We cannot know all the tests in advance. Ideas will come as we explore. If the time for testing gets squeezed, the scope of testing also gets squeezed. Thus, testing that was planned may not get done. Flexibility becomes vital in these situations. Higher risk areas will be grouped and they will likely be performed first.

4. Keep It Practical

Our test strategy should be product specific. Start each test plan with a blank sheet of paper. If you must conform to some corporate standard, try to limit it to a simple outline. Focus on empowering the team to do good testing and returning good information to the customer. Incorporate Risk in your planning to focus on the most important parts of a system. Prioritize which areas need the most attention.

5. Empower the Team

If we do not directly and explicitly empower the test team the likelihood of their being empowered during testing will go way down. Focus on People and rapidly create value for the customer, and understand what the customer actually wants. The primary goal of a test plan is to empower good testing and communicate to other stakeholders how we do that.

6. Create Fast Feedback Loops

Do a little testing, get the results to the development team and decide what further testing is needed and in what order. Get these results quickly, and have the envelopment team require feedback quickly. Face to cafe communication is critical at this stage, and a quick turnaround for testing and associated feedback is essential.

7. Iterate Often

While this often happens by default anyway, it's critical that the testing proves be allowed to work closely within development. If issues are discovered, quickly stop testing. Have the project go back to development, and have them make necessary changes. Get the project back into testing. Include any changes and modifications to the underlying code into your test plan.

8. Communicate Face to Face

Face to face communication is essential. Non-verbal cues such as body language, facial expression, breathing patterns, nervous ticks, etc. come through in face to face communication in a way that email or IM never will capture.

Testing is never able to flow on a perfect flight plan. For that matter, neither do planes. We may be off course on an airplane trip 90% of the time. The reason we get to our destination is that the flight crew consistently adjust the flight plan. A Nimble test plan takes this same consideration to heart; we can't know everything, so let's start with what we do know, and through the process of discovery, fill in the blank areas of our map. If we do this, we will be more likely to test that which is important, and jettison that which is not.