Wednesday Book Review: Surviving the Top Ten Challenges of Software Testing

Through the years, I have come across a number of books that I have used and valued. These may be new books, or they may be older ones. Each Wednesday, I will review a book that I personally feel would be worthwhile to testers.

For many people, there is a book that gives them an “a-Ha!” moment, one where they can see that, hey, this book was written just for me. Well, OK, there’s really no such thing as a book written just for a single person, but I will say that this particular book has been one of the most specifically helpful to me.

William Perry and Randy Rice wrote the book “Surviving the Top Ten Challenges of Software Testing: a People Oriented Approach” back in 1997. Quite a few changes in software technology and testing practices have happened since then, so there are a few areas that are not covered. An updated version would be helpful (and Randy Rice says he is working on an updated version of this book, so I’m definitely interested in seeing the changes when it becomes available), but even after 13 years, this book holds up well and sounds very much like the environments many testers work in today, and many of the aspects described in this book are every bit as relevant in 2010 as they were in 1997.

In “Surviving the Top Ten Challenges”, Perry and Rice take on the area that I consider to be the most challenging when it comes to testing; interacting with other people and groups within an organization. Perry and Rice take on the issues that they saw time and again when they would consult with companies and help them solve issues related to software quality, systems, and yes, testing. As they state early in chapter one, testing is “challenging enough in and of itself, but when coupled with office politics and interpersonal conflict, it can task a tester’s mental health”.

The book uses a unique format - the ten challenges each get a chapter to themselves, and each chapter is formatted the same way. In a way, this allows the reader the opportunity to focus on the challenge that most concerns them (not all ten challenges may be facing each tester at a given time, but it’s a pretty good bet that at least one of them will prove to be here and now relevant).

Here's an outline of how each chapter is set up:

- Overview
- State of the Practice (a worst case scenario situation that describes the issue for illustration purposes)
- Impact on Testing
- Solutions to the Challenge
- Solution Impediments
- Guidelines for Success
- Plan of Action

In descending order are the original ten challenges that were presented. Randy has since said that, were he to write this book today, he would reorder some, and perhaps remove some and add new ones that have come to the fore in the past 13 years, but overall, I think these are still representative of many organizations (your mileage of course may vary). I’ve also added my comments to each of the challenges and how I think they are still relevant today.

10. Getting Trained in Testing (not so difficult to do today, especially with the proliferation of information available through Wikipedia and other sources, but it still requires a fair amount of self-initiative to do. Getting buy-in from management to cover formal test training and pay for it is still an issue in many organizations).

9. Building Relationships with Developers (this is very important, and is still a source of friction in many organizations. Working in a non-adversarial method is vital to the success of any long-term involvement in Quality Assurance and Testing).

8. Testing Without Tools (today, the explosion of open source software testing tools makes this less of an issue, but the adoption of tools or standardizing on tools is still an issue. By the way, for those looking for tools, Randy maintains an excellent list of "Cheap and Free Test Tools" at his Rice Consulting site).

7. Explaining Testing To Managers (I’ve found that I haven’t had to explain the need for testing, but I’ve definitely had to explain what levels of testing are needed, and occasionally to talk someone off a ledge thinking the situation was hopeless, and that there were solutions we could use that were not going to require everyone to become ascetics monks for six months).

6. Communicating With Customers and Users (if I were to order these based on 2010 realities, I think that this would be the number 1 challenge, and the one that many organizations find the most difficult in this day and age. Amazing that with increased availability of communications tools, we are still having conversations about how best to communicate, but there we are).

5. Making Time For Testing (an issue then and an issue now, coupled with having enough dedicated testers and resources).

4. Testing What’s Thrown Over the Wall (there are still quite a few shops out there that practice a strict waterfall methodology, so this is still an issue for many, but adoption of Agile methods by many companies and work groups has made this less prevalent in the last decade. In some companies I’ve worked, though, development takes place off site and code was literally delivered via Fed-Ex, so there’s still the reality of “over the wall” testing happening. Deal accordingly).

3. Hitting a Moving Target (this situation is still prevalent, and it’s still a way too common occurrence).

2. Fighting a Lose-Lose Situation (the lose-lose situation Perry and Rice are referring to is the classic conundrum “find all the issues” coupled with “don’t delay the schedule”. It’s a Catch-22, much of the time. The best solution is to make sure that you do not present yourself as the “guardian of the gate” but rather the “teller of the story” so that those who are the stakeholders can decide what they want to do. This way, developers and project managers are the ones making the choice to go ahead, not the testers).

1. Having to Say No (really being the bearers of bad news; remember, test doesn’t have a veto power on a release any greater than any other stakeholder and key player, but we have an eye towards the issues that would potentially really irritate a customer, and we have to report that reality do or die. Of course, how it is reported is key. We are on the same team as the developers and the project managers, and we all have the same goal, to release a high quality product to our users. When approached with respect and with tact, but also with a full accounting of all testing and efforts, and our honest interpretation of all data, we can all come to a conclusion, and if testing is overridden, we have done what we had to do.).


There is also an additional chapter at the end of the book, “Plan of Action to Improve Testing”. In this section, there are some concrete plans that the user can address when they want to take on any of the challenges mentioned in the previous chapters, as well as ways to foster change in your organization to encourage that these areas get addressed and rewarding those who make the initiative and follow through with the desired changes.


Bottom Line:

It’s 13 years old now and some of the challenges presented are not the same as those faced by organizations today, but people are people, and every one of these areas could be potential challenges, and each one of the methods for improving the issue can be implemented to, very likely, much positive encouragement by any organization. Again, not every one of the situations will be relevant to everyone, but each situation will likely be relevant to someone in a testing organization somewhere. Even if just one of these challenges is met, addressed and dealt with, this book would be worth the read. As it is, it’s one that will be a permanent part of the “people skills” section of my library.