BOOK CLUB: How to Reduce the Cost of Software Testing (3/21)

For almost a year now, those who follow this blog have heard me talk about *THE BOOK*. When it will be ready, when it will be available, and who worked on it? This book is special, in that it is an anthology. Each essay could be read by itself, or it could be read in the context of the rest of the book. As a contributor, I think it's a great title and a timely one. The point is, I'm already excited about the book, and I'm excited about the premise and the way it all came together. But outside of all that... what does the book say?

Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry. Today's entry deals with Chapter 2.

Chapter 2: The Cost of Quality by Selena Delesie

In Chapter 2, Selena makes the point that Quality is always a cost. Good customer service is a cost, solid marketing materials is a cost. Salaries for hiring excellent developers and testers is a cost. Training, fixing errors, testing (and retesting) and reporting... these are all areas that an organization needs to be on top of their game to deliver quality... and none of them in and of themselves makes money. But taken together with a solid product offering that is clean, well designed, and well tested, the potential to earn money goes up considerably, much more than if those steps are not performed. This is the cost of quality.


If we ask ten different people in an organization what "Quality" is, we'll get some good ball park answers, but we probably won't get a solid consensus on what that term means. It's subjective, and it's situational. Quality craftsmanship in things like furniture or in jewelry might be easier to spot, because these are areas where an aesthetic comes into play. We consider these items to be high quality because we appreciate how they look or how sturdy they are. Some of this filters into software as well, but the metaphor fails a bit because software is not a manufactured good in the literal sense like a chair or a automobile door panel. It's much more malleable, and the environment in which it works is capable of changes that a chair or a door panel is not subject to.


My favorite description of quality is one that Selena references as well...    “Quality is some value to some person.” - Gerald M. Weinberg


So ultimately, for a product to be seen as a quality product, it must first be seen as valuable. If there is no value to a person, then the quality of the item is next to meaningless.


A common refrain in small companies that don't have dedicated testers is that the developers work on code an do their own testing. They have unit tests, they have code coverage tools, they check on each other, so the role of testing is already covered. I currently work or a company for which this was the case. Earlier this year, they decided to hire a dedicated tester... me :). There was definitely a change in perspective and while I cannot say that I was a magic bullet that transformed their work from dreck to amazing (frankly, even without a tester, their TDD approach was pretty solid), I did bring a different perspective and approach, and that helped me point out areas the developers hadn't considered (or put more clearly their reasoning why they did things the way that they did). My point is that, even when teams are highly functioning without testers, the addition of one or two testers can help spot issues before their customers do, issues that they might never have considered to be issues.


The next logical jump is to think that testers will test quality into a product, but that's not how it works. Testers are not creators in the literal sense. We can tell what software is doing, but we are not the ones that make the software do that. The developers are the makers, and so long as the developers do not take on the challenge to approach solving the problems or issues in the code, the testers efforts will be a cycle of bug reporting and fixes. For Quality to truly rise, development has to also approach the processes they use and consider where issues appear and why.


Agile development has become a common catch-phrase in various organizations, but Agile in and of itself is not a silver bullet. It is an approach, with a variety of techniques, that help teams communicate better, streamline process guideline, allow for enough documentation (but just enough) to help teams be effective, with an emphasis on Test Driven Development, pairing, short iterations, and slicing the development areas into thin segments to be focused on. it's fast, to be sure, and if practiced by all team members effectively, it can be a very good development model, but it no more guarantees higher quality than any other software development method.


So how does this all come together? Testers alone don't ensure quality. development practices alone don't ensure quality. Tools and test driven development don't ensure quality. A process manual or index cards don't ensure quality. An engaged and passionate customer support team reporting issues and concerns don't ensure quality. Non o these things by themselves do, but when all taken together and all areas worked on as a fully "holistic" and "systemic" approach, *then* quality really does have a chance to develop and grow. For quality to become more than a buzzword, though, it has to be embraced and practiced by *everyone* in the organization. It can't just be a buzzword. It must be a real and active part of everyone's day to day practices.


Quality doesn't magically appear. It takes work and commitment, and yes, it's an expense. Still, the cost of not implementing and utilizing quality standards effectively may be far greater than actually implementing and living by them. Also, there is no one size fits all quality approach. every organization is as unique and different as the people and personalities that make up that organization. In a way, I liken quality to the quote attributed to Derek Bok... "if you think education is expensive, try ignorance". Likewise, if you think quality is expensive, consider what is happening without the quality improvements that are possible. Selena's not saying it will be easy, but with the right organizational approach, it can prove to be very much worth it, perhaps many times over.