Wednesday Book Review: Agile Testing

Through the years, I have come across a number of books that I have used and valued. These may be new books, or they may be older ones. Each Wednesday, I will review a book that I personally feel would be worthwhile to testers.

In many ways, this book requires two reviews. One would need to be for the reading and the information imparted, the other would be for the applicability of the principles. As the organization I am working with is just now embarking on its first full scale Agile project, I will have to save the applicable review for later, as to be frank, I just do not have the experience level to speak to them as of yet. So today’s review will have to be to the readability and ability to comprehend the ideas that are in Lisa Crispin’s and Janet Gregory’s Agile Testing: A Practical Guide for Testers and Agile Teams, and to that end, this is an excellent title to dig into and learn about Agile practices.


At 576 pages, there is a lot of material to digest, but the pace of the book and the breakup of topics allows the reader to cover a lot of ground quickly. The book begins with an explanation of Agile goals and responsibilities, and also includes the “minor heresy” of removing the context of the testers as a stand-alone unit. Make no mistake, for those who have worked in traditional development environments, this book may well come as a shock to the system. Lisa and Janet make the point that the testers do not stand apart from the development team. In fact, they are directly integrated into the development team and stand right next to the developers in scope and impact.


Each chapter beings with a “mind map”, which helps to organize the sections to be presented, and in many ways, readers can look at the mind map graphics and get a very quick understanding of what the next chapter will cover. This is by design, and it is meant to instill the idea of breaking away from strict, structured documentation that, while complete, is often ponderous and time consuming. By contrast, the mind maps allow a quick scan to see what the main points of the section will be, and then with that information, the reader can dive in and follow along.


Agile Testing is broken up into six areas.


Part I introduces the reader to the roles and activities of an Agile team, and defines the roles and responsibilities of a Customer Team (everyone who helps define the business aspects of the product and how they will be implemented) and a Developer Team (how to determine the method in which the customer’s objectives are actually met), As tester’s, we are firmly planted in both teams, and are expected to contribute to both. Lisa and Janet make it clear that the role of an Agile Tester is to not act as the “gatekeeper of quality”; that role belongs to the entire developer team, and we are part of that process. The ones who decide on the quality criteria are the customers. The key to Agile for testers is that we get engaged early, and our testing efforts drive the project forward with steps made in a much more rapid fashion than traditional development environments. In an Agile environment, everyone is a tester, whether they write code or not. The defining attribute of a successful Agile tester has less to do with skills than with attitude, one that embraces change, experimentation, and collaboration.


Part II describes the organizational challenges of a team making a shift from traditional development methods to Agile. There are often large scale changes that take place, and for many people, long established roles and scopes are often moderately tweaked or completely thrown out the window. Giving up the idea of an autonomous and totally independent testing team can be a jarring experience, but it doesn’t have to be. It’s also not just the testers who go through a period of adjustment and re-calibration. Developers, project managers and executives often have to get past comfortable niches to embrace an Agile environment, and adapt to a faster and more focused work schedule. The change can be chaotic and scary. Being willing to work with that fear and try new things is one of the primary keys to success. Most important is coming to grips with the fact testing is not treated as a separate activity that occurs after development, but that testing and coding are integrated activities and happen side by side. Processes are treated differently in Agile projects. Unlike traditional gated development methodologies which have heavyweight process documentation, Agile takes a different approach. It doesn’t do away with the need for test plans, requirements documents, bug tracking or auditing requirements, but it approaches them from a different angle, maximizing effective time and effort on the tasks of developing quality software, and providing enough documentation and process to get the job done (with a hearty emphasis on the "enough", as in no more than :) ).

Part III introduces the Agile Testing Quadrants, and these areas of testing appear in the subsequent chapters. This area focuses on The Story and how that story helps to drive both Technology-Facing Tests and Business-Facing Tests, and how The Story helps to develop both the underlying code requirements and the necessary tests. The Four Quadrants becomes a consistent theme for this section, and using each of them (Q1: Unit Tests and Component Tests; Q2: Functional Tests, Examples, Story Tests, Prototypes, Simulations; Q3: Exploratory Testing, Scenarios, Usability Testing, User Acceptance Testing, Alpha/Beta Testing; Q4: Performance & Load Testing, Security Testing, other “ility” testing) and the context of each set of tests. The tools required to perform these tests are also described, and some examples as to when they are used (Test Driven Development, Continuous Integration, source code control, etc.).

Examples and business-facing tests are emphasized, with the idea of working on small areas in short iterations, to give the customers a chance to see how the product is developing and get real time use of the application and offer feedback for the next iteration. In this stage, working with the team to break up feature into smaller and more manageable stories is presented, with an emphasis on what Crispin & Gregory refer to as “write test-write code-run test-learn”. Some example tools are shown to help organize the Agile tester to prepare to develop stories and convert then to requirements and tests, ranging from simple index cards and paper models to using spreadsheets and full feature test applications such as Fit and FitNesse. An emphasis and an excellent description of Exploratory Testing and practices associated with it are explained. This may be a surprise to some who felt that Agile was all about automated testing. Crispin and Gregory make very clear that a live person paying attention to the details is critical to the success of an Agile project.


Part IV goes through the process of automating tests associated with an Agile project. The fact that it is necessary is a given, but getting started is often the biggest challenge for some organizations. Barriers to successfully implementing automation are discussed and strategies for implementing automation in an Agile environment are covered. Crispin and Gregory make the case that using the four Agile testing quadrants will help identify where test automation will be most needed, and when. Focus on repetitive actions wherever possible, specifically when you perform builds (continuous integration, anyone?), unit tests, functional tests, performance test, etc. It’s also important to realizes that automation is good in key areas, but it’s not a silver bullet. There are areas where real human interaction provides more value than the promise of automation will.


Part V covers the testers role in the life of an Agile test iteration and provides lessons learned to show where testers add value and when. Testers definitely have an early hand in the development of user stories and using the time to clarify and focus on these stories and developing test cases around them is critical early on. Planning for the upcoming tests is also important, whether that be prepping environments, implementing test scripts (manual or automated), or gaining better clarity of the scope of testing can be time well spent. During the iteration planning session, testers can get ready by making sure that task and test cards are written along with development task cards, and given an realistic assessment as to how long those tasks will take. Once coding begins, the tester gets involved with quick iterations of test-code-review-test increments.

When problems arise, use the "Power of Three" to help resolve the issue (Power of Three being get three different viewpoints from three different people before making a change or committing to a course of action). Add automated tasks where feasible, and allow time for manual exploratory testing to help fill in any gaps in both test and coding coverage. At the end of the iteration, participate in the review to see what went well and what could be improved upon. Focus on one or two issues at a time. Celebrate successes large and small, and look for ways to resolve issues with testing for the next go around. When the product is ready for release, make sure to cover the bases regarding release, documentation, installers, upgrade scripts, and any other elements that will be part of the final release. Be sure to test database update scripts, data conversions, and other parts of the installation.


Part VI wraps it all up, and gives the factors that Crispin and Gregory feel are key to success with Agile practices. They emphasize that Agile testing is not separate from, but a core part of, the development process, and that the tasks and responsibilities of the Agile testier are integrated directly with the efforts of the development team as a whole. The goal of the tester in this new realm is to be a proponent for continuous improvement, to champion doing better each go around, to work to create a successful collaboration between the developers and the customers, and to always look at the big picture, which is to make sure that the best quality product is delivered to the customer and that the development effort improves with each test iteration.


There's so much to this book, and the space of this review can hardly do it justice, but Agile Testing does a good job explaining the differences in an Agile environment, and the way that a tester will interact with an Agile team. For testers who do not come from an Agile background (raises hand), there was much in this book that I found to be both helpful and encouraging. One of the biggest positives to this book is the personal experiences provided throughout from Lisa, Janet and other contributors; this adds a much needed perspective to the theoretical ideas being provided, and helps to give a better overall picture of exactly what I could do in similar situations (or not do in certain cases).

Bottom Line:

If you are new to the world of Agile, this will make for an excellent primer and blueprint for determining how to integrate yourself into an Agile organization. With the fact that I am just now getting my feet wet with an Agile project, it’s been very helpful in giving me ideas and methods to approach testing and how to make myself useful at all stages of the development iterations. While it’s too soon to tell if all of the suggestions will work for my specific environment, I’ve already been able to make some clear advances in my way of thinking about working on my latest project. I’ve also replaced quite a bit of apprehension with a sense of excitement and anticipation. I’m impressed by what I’ve read; now I’m looking forward to putting its lessons into practice.