Wednesday Book Review: Software Test Automation

Through the years, I have come across a number of books that I have used and valued. These may be new books, or they may be older ones. Each Wednesday, I will review a book that I personally feel would be worthwhile to testers.


In a previous review I said that I had two books that were my only test related titles through my entire career (at least my career up through 2007). Testing Computer Software by Falk, Kaner and Nguyen was one of them, and this book, Software Test Automation, by Mark Fewster and Dorothy Graham, was my second. Though this book was published a little more than a decade ago, I still refer to it. Is this a title that others should read, too, in light of the fact that ten years of progress on the automation front has taken place? Why would I suggest this book?


In the days when I was working in a predominantly UNIX shop, the ability of writing code and writing test scripts for that code was almost the same thing. The ability to use tools like bash, awk, and expect meant that most automation needs could easily be put together for many applications. However, with the coming of graphical interfaces, the simpler tests were more difficult to conduct (or in some ways impossible). In the mid to late 90’s a number of automation tools were developed with the promise of recording your actions, and then playing them back over and over. Well, most of us who have had any run-ins with automation know that that was mostly an empty promise, and even in the best of circumstances produced tests that were too brittle to be of any long term use, and even those that were required a lot of maintenance. Fewster and Graham point this out very early in their book, and show that automation requires more than record and playback. It requires forethought, good planning, and understanding the limitations of test tools. With that understanding, the authors go through many details and examples to show a "tool agnostic" model for developing tests and applying them at different stages of the software development life cycle.


Many test automation books require the reader to have a background or at least a good understanding of programming. Fewster and Graham make no such assumptions, and this book is a great first step for those who want to get started with automation, or have the task of starting from scratch with their automation efforts. I’m somewhere on that early continuum myself; though I understand many aspects of the automation process, the methods that stood me well in a UNIX shop aren’t as readily usable in a Windows development environment.Still, the methodology suggested is still relevant and can help the budding automation tester ask the right questions, even if they don’t know many of the environment specific answers as of yet.


The book is split into two parts. The first section is the introduction and practice of test automation. Eleven chapters bring the budding test automation engineer from initial automation contexts, using the V model for constructing different tests for different stages in the product lifecycle, and coming to grips with the limitations that automated testing has (interestingly, the limitations in 1999 are very similar to the ones experienced in 2010). Subsequent chapters discuss techniques of automation, including various methods of comparison and approaches to determining if tests have passed or failed, and how to really know if they have. A focus is placed early on making sure that tests can be maintained, and a methodology for constructing tests so that they are not overly complex is a highlight of the book. Further chapters deal with metrics and measurements, monitoring and reporting results.


The second part of the book goes through a dozen short chapters that show case studies of companies that have implemented automation, some successfully, others not so successfully. Microsoft is also included, describing how they implemented test automation in the 90s, specifically the initiative that came to be known as WebTV for Windows), and relate their successes (and failures) with real-world test automation. Modern readers may look at a number of these chapters with a sense of amusement; the examples are mostly derived from the early to mid 1990’s, and do not have many of the common buzzwords that we are familiar with today. This book also predates the Extreme Programming and Agile movements, so there’s no mention of those approaches. Still, even without those newer flourishes, there are many great take-away’s from Fewster and Graham’s book.

Bottom Line:

Software Test Automation is two books in one. The first book is a how-to guide to get into test automation and develop a method of constructing automated tests and thinking about automation that will help place the tester on a solid footing to grow and experiment with. The second book is a glimpse into a number of tester’s real world issues and answers to testing challenges, and hoe they met them (or didn’t meet them). The details in Software Test Automation, were they to be laid out end to end, would be able to keep a tester busy for several months, if not longer. Don’t let this discourage you, as the book is very readable, and it’s a title that allows and encourages jumping around. For the novice automation tester, it’s a great foundation to build on.