My Quality Assurance Toolkit

Just as a carpenter would never dream of going to a job without his tools, we also need to decide what our tools are and how we can best utilize them. Different jobs require different tools. Some places are closed shops and only use proprietary tools or things developed in house. Others are more flexible and encourage testers to get out there and try new things.

Here is a sampling of my toolkit, and I will expand on each of these at a later date.

1. Test Plan Templates: Not everything is going to be tested the same way all the time, and you may likely never use the same exact test plan twice, but almost all test plans share things in common, and having a few ready-made templates to use for certain circumstances can cut down on the time needed to write a first pass test plan. I prefer using either Excel or a wiki tool to write test plans due to their speed and ready availability to others to review and make comments. Excel has another benefit that dovetails into the second part of my toolkit. It's easy to chop up test plan pieces to integrate into a "test dashboard".

2. A Test Dashboard: Again, the implementation will vary depending on where you are at and what you are doing, but in most cases, a simple dashboard will give you a heads up in how to communicate what you are doing to others in your organization. Dashboards can be elaborate or they can be simple. What matters is that you are communicating the relevant information in them and that your data tells a story about what you are seeing. I personally prefer a low tech method for dashboarding, which is to use an Excel spreadsheet model that I was turned onto a few years ago. It aggregates a number of items from manual test cases and scripts and then lets the user compile all of that data into a simple view. The other benefit is that this is a way that you can get rapid reporting and shows what you are doing while you develop a more sophisticated method later on.

3. Cygwin: This is a personal choice, but one that I do not like to go without. For many years I worked in a primarily UNIX shop, and I grew very accustomed to the methods and approaches to scripting in a UNIX environment. While I enjoy using UNIX and Linux, and keep some machines around to play with, there have been times where I have worked in purely Windows shops, and while Windows has a number of useful tools, there’s something to be said about the quick and direct elegance that UNIX pipelining of utilities provides, so cygwin is my simple compromise. If you have no familiarity with UNIX, then these tools may not be much of a help to you, but if you have worked with UNIX and wished many times for the ability to use awk, or like the direct scripting interface of the bash shell, or just find yourself way more effective editing text files using vi, cygwin bridges the gap very well.

4. A Scripting and a Development Language: Actually, there may be many languages you’ll need to know to be truly effective, but even if you only know one scripting language and one programming language really well, that can stand you in very good stead (and also give you a leg up on working with other languages). For me personally, I’m most familiar with C on the coding side. The shop I work in currently does most of its new work in C#, so there are a lot of familiar aspects between the two. In the scripting world, Perl is the language I am most familiar with (while I used to do a lot with Tcl/Tk, it’s been a decade since I used it in any meaningful sense). Still, having a background in Perl helps me to see common patterns in other scripting languages, and then just looking at the details helps me to fill in the blanks where needed. Again, getting up and running with scripting isn’t hard, but getting proficient with any language (programming or scripting) takes time and effort to get really good.

5. An Active Library Card: Some may scratch their heads at this one, but I’ll explain why I consider this to be important. The Peninsula Library System in the county that I live has 34 interconnected public and community college libraries; the ability to get access to books on just about any discipline fairly quickly is very valuable. However, the true gem in my library card is the subscription that it provides to Safari Books Online. There are hundreds of books related to software testing alone, and thousands of books related to programming, software engineering and any other number of topics, all available online and with the credentials provided by my library card.

6. Some Open Source Test Tools: What you use will vary from where you work and what your focus is, but I will recommend highly that developing some familiarity with an open source tool or a number of open source tools will give you and edge and provide you the ability to test things “right now”. These tools can range anywhere from basic macro recording tools for applications to sophisticated web based capture/playback tools rivaling features offered by the professional offerings. What’s more, most of these tools can be downloaded and used with little or no cost to you or your organization, and the skills you develop using these tools will many times translate over to commercial tools, too. Examples I currently use are twill, selenium and watir, but there are hundreds if not thousands of open source offerings to choose from.

7. Some Basic Reference Books:“Testing Computer Software”, written by Cem Kaner, Jack Falk and Hung Quoc Nguyen. I’ve had this book for well over a decade and still refer to it often. Another title I’d definitely recommend is William Perry’s “Effective Methods of Software Testing”. These two books together will provide a tester with lots of good practices and excellent ideas to apply to testing and help develop a lot of great skills. In addition, I would recommend getting books that relate to the business of the product you are testing. Many times we can be well versed in testing skills but without knowing the lay of the land of the particular business that we are dealing with, we will be much less effective than we could be (currently I work with software that helps with Case Management for Immigration Attorney’s. Due to this, my primary direct domain knowledge book of choice is “U.S. Immigration Made Easy” by Ilona Bray). Having worked in test positions related to virtual machines, networking equipment, capacitance touch devices and video games, I have owned and read books that are relevant to those spheres as well. Having a few titles to read and help you understand the domain that your software product is meant to work is the important part.

8. A Virtualization Library: While not every testing role or environment will be suited to being tested in a virtual environment, many applications can be tested this way, and having the ability to build and update environments on the fly can be a huge benefit. There are pluses to having commercial grade tools like Microsoft Hyper-V, but even without that level, users can create virtual machines easily and for free with tools like VMware Player, VMware Server and Virtual PC. Also, creating test environments with demo versions of products is a great way to test environments without having to purchase entire suites of operating systems and applications (the 120 to 180 day Evaluation copies are perfect for test environments, since the likelihood of an environment being around that long before needing to be reinstalled is pretty low, and if it will be around that long, probably justifies purchasing a license).

Is this *everything*? Not even close, but it’s a good representation of tools and techniques you can bring into any testing project, and be able to hit the ground running fairly quickly. Also, there’s no way to make it so that there is a simple “one size fits all” toolkit for every purpose, but the good news is that, at least with software, techniques are malleable and can be adapted fairly easily. At the end of the day, though, you will be the one that needs to create your own map and approach things your own way. These tools have helped me, perhaps they can help you as well. In the future, I plan to talk to each of these areas a little more in depth, but I’d also like to hear what tools you consider important in your testers toolbox.