BOOK CLUB: How to Reduce the Cost of Software Testing (8/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?


Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry.


We are now into Section 2, which is sub-titled "What Should We Do?". As you might guess, the book's topic mix makes a change here. We're less talking about the real but sometimes hard to pin down notions of cost, value, economics, opportunity, time and cost factors. We have defined the problem. Now we are talking about what we can do about it. This part covers Chapter 7.


Chapter 7: Test Readiness: Be Ready to Test When the Software is Ready to be Tested by Ed Barkley


Ed starts the chapter with a painfully familiar refrain. Many times, we are ready to start testing, but our environments are not, or they are not in sync with the development machine, or needed patches have to be applied, or other server requirements have to be rolled out so that testing can begin. after a much protracted comedy of errors, the test team is ready to hi the ground running... a week and a half after testing was supposed to begin.


Sound familiar? It sure does to me, I've lived this more times than I care to admit, at more than a few companies. There is a beautiful symmetry to the Boy Scout Motto and a test environment actually being really and truly ready to go. "Be Prepared" is more than a slogan, it's really the best way to hit the ground running when you really need to be doing so.


Good planning can save you money! It's one thing to plan your work and work your plan. It's another to anticipate issues and be ready with a contingency plan at the moment an issue arises. Having backups, servers in reserve, virtual environments that can be quickly spun up in the event of a crash, etc. will help you to be able to stay ahead of the curve. Cool. Can we actually do that? Ed thinks the answer is yes :).


Having test cases and being ready to test is just the tip of the iceberg. To get ready to "catch the wave", you'll need a little more in your quiver.


Areas of Concern to be addressed first are your Test Environments themselves. Are they up to date? Do they have the latest OS patches, application patches or anything else they need to do what they need to do? Next comes your Configuration and Change Management. Is the hardware, software, firmware, data and databases, documentation, test tools/fixtures, and test documentation in sync with what the developers are using? Are you sure? Is all your Documentation up to date? That means Requirements docs, Test Plans, Traceability Matrices, Models, Use Cases, User Stories, Test Scenarios, Cases and Scripts, whatever docs you use. Is your Data up to date? Is your "Golden Database" really golden? Is your test data you'll be using in sync with the Golden Database? Are your Test Tools up to date? Have you validated your automated scripts, your macros, or other computer aided testing tools? Do you know who your Testers are? Have they been trained in the tools and scripts they will be using? Is your issue Issue Management tool up to date and ready for use?


Think of any of the above areas. Is there a possibility something could happen in any of those spheres? If so, what would the impact be? Knowing this can help you plan for contingencies or any other number of options to get you back to up and running.


It's likely that anything happening to the Environment, Configuration and Change Management, Data & Testers will have a high impact on a project. Issues with documentation may warrant a moderate impact, depending on when the change or disruption occurs. Earlier in the process is better, later can eat up a lot of time for things that are not needed, or not allow for time for things that are important but unknown. Issues with Test Tools & Issue Management may be moderate to low. Although losing the speed of automated testing, we can still test, and sometimes we can cover more ground manually (if we are using exploratory methods) than our automated tests can in their well worn grooves :) ).


Ed recommends developing a checklist to help you determine if everything is ready to go. Just like I as a Scoutmaster use a checklist to make sure I have all the information that I need for my Scouts any time we do an outing like summer camp, the checklist can be just as important for the start of a major software test cycle (or any test cycle for that matter). To see the full list of checklist options for each area, check out the book :).