BOOK CLUB: How to Reduce the Cost of Software Testing (16/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry.

We are now into Section 3, which is sub-titled "How Do We Do It?". As you might guess, the book's topic mix makes a change yet again. We have defined the problem. We've discussed what we can do about it. Now let's get into the nuts and bolts of things we can do, here and now. This part covers Chapter 15.

Chapter 15: Clean Test: Suggestions for Reducing Costs by Increasing Test Craftsmanship by Curtis Stuehrenberg

Curtis opens up this chapter with the idea that we can learn a thing or two (or more) from the Software Craftsmanship movement. Software craftsmanship is the idea of an experienced coder developing their skills and techniques over years of study and practice. Curtis expands this description by saying:

“A software craftsman is a skilled artisan apparently able to balance immediate pragmatism with a longer term focus on reducing the amount of work they or (more likely someone else) must do tomorrow. The craftsman coder knows a small investment in time today can save days or weeks later on, but more importantly they’ve developed a sense of which small investments will reap the greatest rewards.”

Software Craftsmanship emphasizes:

• Not only working software, but also well-crafted software
• Not only responding to change, but also steadily adding value
• Not only individuals and interactions, but also a community of professionals
• Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

Notice that the above highlighted areas are not specific to software developers. These ideas and ideals apply just as much to testers as they do to developers. Having to fix someone else’s code, or even your own code, over and over, prompted the inspiration and the growth for the “Clean Code” movement. The idea of Red-Green-Refactor is part of the view and ideal of “always leave the campground cleaner than they found it.” The ideas behind Software Craftsmanship are appealing to many developers. They are an underpinning of the Agile movement. Organizations world over are trying these ideas out and making them a core part of their work. Yet where are the testers in this paradigm shift?

Craftsmanship should matter to testers every bit as much as it should matters to developers. There are numerous benefits to well crafted tests; less debugging, maintenance, and rewriting. The function of a software tester is to communicate the experienced behavior software when compared against the expected behavior at a specific point in time.

That’s it.

The tester’s primary role is communicating. Information provided by software testing are reports showing the behavior of a product compared to its expected behavior at the time it’s observed. This information is then used to inform decisions with regards to management and budgeting of the project. This information ultimately helps the decision making process on whether or not to release a product to the customer. A good test plan provides information on what the team thinks is valuable and what constitutes a risk to a project.
Test cases are a conversation the test team has with itself. The danger is that we often fall into a form of short hand to describe what we do and create test cases that satisfy document standards rather than realistic testing needs. Bad test cases often come from being rushed, needing to fill details for documentation or compliance needs, and bad test cases don’t just fade away. They have to be deliberately removed or reworked in most cases.

We can look at the cost of performing an activity and the cost of not performing an activity. This is referred to as “Activity Based Costing”. Ultimately, there’s just one cost, the cost of the entire system. Activity-based costing integrates several procedures—value analysis, process analysis, quality management, and costing—into one analysis. Activity Based Costing includes both “Active Costs” and “Passive Costs” into the “Total Cost” of a product. Managing the whole value for software mean we focus on the entire lifespan of a product or company.

We move beyond the individual release and look long term to the life of the product (or the company itself). Software craftsmanship advocates designing and writing features that are easy to understand, support, adapt, and use both now and later when someone else needs to. Test craftsmanship is the art of doing the same for test cases and testing properties. Historically, testers have been at the end of a project, and usually under tight time constraints. For that reason, the idea of test craftsmanship often takes a back seat to the pressing need to just get stuff done and get it done fast! The net result, of course, is bad and poorly crafted tests.

Good software tests, when implemented, help reduce the various activity costs associated with testing. Good software tests aid communication between team members, help new testers get involved quickly, and have many opportunities for reuse. They do of course take time and analysis to make.

Ultimately, testing craftsmanship can be summed up in the following list (for detailed analysis of each, hey, read the book ;) ).

1. Plan the appropriate testing for the task.
 2. Know your audience and keep them in mind at all times.
 3. Don’t rely on someone else to clean up after you … even yourself.
 4. Refactoring and redesigning are not tools for excusing halfway effort.
 5. There will be no second chances to get it right due to the escalating technical debt you’re currently compounding.

Bad test case craftsmanship and bad code craftsmanship are probably the two most unnecessary costs incurred as part of doing business by software companies today. The software craftsmanship movement is taking on the developers in this regard. Testers, how about we pick up the charge and agree to be held to the same standard?