An Experiment: Teaching My Son How to Code and Test

My son has proven to be a typical millennial kid. Tools like cell phones, texting, using tools like Facebook and Gaia.com are second nature for him. He has been using a computer in one way, shape or form since he was about 3 years old, and his schools have had him making projects using tools like Power Point and HTML for several years.

As I came to see how frequently he was using a computer for social tasks, the thought came to me that he might have some insights as to how to interact with sites and seek to improve them. As a sideline, I do some web development work and write scripts in Perl to handle things like registration forms, payments and the like. As it is, I’m intimately familiar with the axiom “coders shouldn’t be the final arbiters of deciding if their code is good or not” (I don’t mean classic “desk checking” or “unit testing”, every developer needs to do that), but when it comes time for testing the functionality, integration and overall user experience for a site or an application, the final word on whether or not a good result has been met shouldn’t be just the developer to decide that. A dispassionate, third party person should be the one to do that testing. It was with this in mind I thought it would be cool to teach my son how to test. As I started looking at what I might do to help teach him testing, I kept coming back to one thought… “If you have a chance to teach the next generation, don’t just teach him how to test, also teach him how to code!” I figured he might have more fun with the testing aspects if he had some coding skills to compare it with (and use).

Hmmm… am I really the best person to do that? Wouldn’t it be better to have him learn from someone else who may have better development chops? I freely admit half of the reason I was drawn to testing was that I’m not an avid coder. I can do it, and I’ve created test scripts and CGI scripts for many years, but to say that I don’t particularly find coding for the sake of coding fun would be fair. So how could I realistically teach my son how to code in a meaningful manner and in a way that would help set good habits for him?

The answer came when I was listening to Scott Hanselman’s Hanselminutes podcast, specifically listening to his show called “Hello World”. This is the title of a book written by Warren Sande and his 10 year old son, Carter, and is meant to be an introduction to programming in a true and meaningful sense. Scott interviewed Warren and Carter and asked about their methods and how the son was able to go from not knowing programming to a level of high proficiency, and to writing a book that did not dumb down the process or pull any punches. I decided after hearing about this book and approach that my son and I would try to do the same for testing. The challenge is a big one… would it be possible for me to break down the steps of testing, and break down the steps of coding, so that an 8th grader could understand them? Even more, could I do this in a way that an 8th grader would learn and thrive with it? To be honest, I don’t know the answer to this. But we talked it over and he’s willing to give it a try with me (he was really interested when I told him that I’d be willing to share some of the proceeds of my contract work with him if he learned enough to make a meaningful and substantial contribution to new development and testing work that I do… it seems money is a good motivation to a 13 year old :) ).

Thus, we’ve decided to set up shop and start practicing. As the book Hello World is going to be our initial template, we will start out with the language the book uses, which is Python (it has the advantage that it’s not a language I have used very much, so we can both learn the ins and outs of the language together). In addition, we have agreed to set up our sessions so that we work facing each other, laptop to laptop, sharing code snippets and ideas, as well as testing each other’s efforts, critiquing what we see, and filing bugs on each other’s work (borrowing a bit from the Agile approach of peer programming and peer testing, and seriously thinking of introducing a Kanban board while I'm at it :) ).

So, will this effort make my son into an enthusiastic tester and developer? Honestly, I do not know, but what I do know is that it will give my son a better window into the world of what I do. I remember when I was his age, my father did the same for me. My father was a doctor, and he often took me along with him when he had overnight shifts that didn’t interfere with school, and I got to wear scrubs and follow him around and help him with things where it was appropriate for me to do so. While I ultimately decided not to follow in his footsteps, I found the process to be great, and I enjoyed the times I went to work with him and he shared a glimpse into his world with me. It’s my hope that my kids will feel the same way about what I’m doing (this offer, of course, being open to my daughters as well as my son).

This could become quite a grand adventure, so we shall see where this journey leads us.

Developing Skills for Web Quality Assurance

A good friend of mine who also works in the Quality Assurance sphere sent me the following:

"Mike: I need your help to be a better Web/ Flash QA dude. I'm currently enrolled in a Flash Actionscript class, and I plan on taking a JavaScript class/ continuing w/ my web dev engineering courses. What else can I do to be a better web/dev QA?"

In many ways, the web is both easier to test and more difficult to test. It’s easier in the sense that many of the options to be tested can be modified on the fly and deployed rapidly, so test iterations can be quick and focused. It’s more difficult in the sense that the platform options are wide open. Operating systems, devices used, browsers and browser iterations, plug-ins, and that’s not getting into the unique challenges of displaying the pages on mobile devices. Testing aspects like Flash and JavaScript apps have their own unique domain knowledge requirements, but just as important are the issues that come with any web application, that of testing the client and the server in the areas of functionality, performance, and, most critically, security.

The web is an interesting development paradigm, in the sense that most transactions, ideally, are stateless. This allows for many users to access a server at scattered times and not bring the server to its knees; web servers can be gauged as to how they handle concurrent connections, so being able to test and confirm that applications run as expected while maintaining a high rate of concurrent connections would be a definite positive step. There are various load testing tools that can help simulate high network traffic and provide the ability to simulate a server under varying loads (I would recommend looking at open source options for these; while they may not be exactly transferrable to certain jobs because they don’t use a particular tool, the skills learned and the understanding of the methods most certainly are).

Learning how to do some client side automation would be helpful. A great tool for doing this if you use Firefox is the Selenium IDE. Selenium has a number of tools that are useful in different areas, but the Selenium IDE is a plug-in that goes right into your Firefox browser and gives you a bunch of options to record and modify tests.

Understanding HTML, CSS, XML, and other markup and scripting languages (ASP, JavaScript, CGI, etc.) all play a role in the presentation of pages. Additionally, understanding the underlying protocol requests and responses is also very helpful. You can be much more effective testing web sites and web interaction if you have a solid understanding of the options associated with HTTP (and there are a lot of them!). You can read the most current drafts and standards for the HTTP 1.1 protocol (RFC 2616). Hand in hand with learning about the HTTP protocol is learning how to work with protocol driver applications. Several are on the market today, both in the commercial and the open source spheres. One that I enjoy using currently is called twill. This is a command line interface that allows you to send HTTP requests and receive HTTP responses in a framework (effectively, verify that a site is returning the proper objects and responses without even firing up a browser).

The area with the greatest risks and the largest potential for issues is security. Security can be compromised in any number of places; in the client, in the web server, in the database server and even through proxy servers set up to capture data. SQL injection can be used to compromise sites and gain access to databases. Social engineering tricks can be applied to see how to get access to sites and to hack into sites… and yes, to get good at doing web Q.A., you need to learn a few hacker skills and know how to apply them, as well as how to prevent them. A book worth reading is “The Web Application Hacker’s Handbook: Discovering and Exploiting Security Flaws” by Dayfydd Stuttard and Marcus Pinto (Wiley Publishing, 2008). Also, for a good and entertaining primer on the topic of Security, I recommend Randy Rice’s Software Quality Podcast Episode 5 (this was recorded in June of 2006, but the general ideas discussed are still very valid, and Randy does a great job of presenting the ideas and issues).

If you want to build as much knowledge about the web as possible and really cover a lot of different techniques, I would be remiss if I didn’t mention W3Schools site. This is a terrific and comprehensive reference site for web technologies, and it gives the users many opportunities to practice the principles right in the site, and make changes and see how the changes work against the example code. This has been my go-to site for many years, and it has proven to be tremendously helpful to me.

In my opinion, the best way to learn and practice Web Quality Assurance is to get together with a few other people, set up a server or two (you can do this easily in virtualized space with tools like Hyper-V, VMware or Virtual PC if you don’t have spare boxes) and design some applications and pages to interact with. From there, take turns trying to see who can bring the server to its knees. Once you have done so, have a debriefing between you and your friends to determine who did what and how, and then look to see how you can harden the server . Repeat the process, letting someone else attack the server and another person's example applications and setup. Note: do be careful to set this up in a way that potential “over-spill” is contained; it’s not wise to try exploits or use tools on open networks, as your end results may go way beyond your area of testing, affect more than just your target machines, and might get some people really mad at you (not to mention some law enforcement interest if you go really hog wild!).

To TESTHEAD readers, do you have additional suggestions? If so, please feel free to add them to the comments section.

Snowboarding and the Challenges of Automation

I had a great weekend with my elder daughter; we went to Donner Ski Ranch near Lake Tahoe to enjoy what has become an annual tradition for us, the Legends of Tahoe Snowboard Competition, Reunion and Party. I took the first half of the day to work on technique with her, and then I spent the latter half of the day split between riding and seeing my heroes from years past still out there doing amazing things.

One of my goals with my kids is to watch them ultimately surpass me. I dream of the days when my kids will be able to leave me in the dust when we go ride, and I’ll have to be the one to play the catch up game. We are not there yet, and each time we go up, I realize that there is certain precedence to getting the skill to ride well. For them at their young ages, there’s a lot to remember and process; I take this for granted at times because I’ve been snowboarding for the better part of two decades and used skis for almost fifteen years prior to that time. When you are first getting your bearings and learning how to get everything to work together, there’s a lot one has to keep their focus on. I’m sure my daughter got very tired of me telling her the same things over and over; “look down the hill towards where you are going”, “keep your hands in front of you where you can see them”… and the phrase I’m sure she wishes she could expunge from her memory because she hears me say it so often… “ride like a boxer”, which means to hold your hands in fists and assume a pugilist's stance; it helps with balance and making sure that your center of gravity is over the balls of your feet as you execute turns. With time and effort, all of these started to come together for her, and by the end of the day, she was turning like a champ.

I mention this because the last few years, due to my kids getting older and having so many other things that compete with their time, and the challenge of corralling three kids to go snowboarding at the same time, it’s required my use of a divide and conquer method, where each kid gets designated days where I ride with them and help them get better. Multiply this out by three kids, and I really only get two days a season with each kid, which means a lot of retreading on basic principles happens each year.

So what does this have to do with testing? It dawned on me that, as I have been working on automation projects, or more to the point, trying to work on automation projects, I have the same challenges my kids have when they snowboard. First, automation is a nice addition to my job, but at this time, I’m not really being paid to automate, I’m being paid to test. To expand my kids’ snowboarding metaphor, they don’t necessarily appreciate the fact that there’s a technique to getting down the mountain, they just want to get down the mountain and have fun while doing it. I of course try to encourage their technique because it will give them more pleasure to ride with confidence. They aren’t there yet. When it comes to me and automation testing, likewise, automating will provide a lot of benefit to certain projects, but really, I need to get the testing done (read: get down the hill, by whatever means necessary). For me, I came to realize many years ago that, in my pursuit of snowboarding, I had two choices, ride free but face awkward situations where I’d have difficulty getting down the steep parts, or spend some precious time practicing, learning, and practicing some more until those motions became automatic. 

Likewise, automation requires time, and in the early stages, a significant portion of it. There are core techniques we need to learn, such as setting up frameworks, making our tests specific, but not too specific, and starting out small and working our way up. My dreams have proven too big for my abilities more times than I care to admit, so I need to remind myself that there are some basic details that must be addressed first. The tools are only as good as those who use them (back to snowboarding, I can have the most fabulous boards, boots and bindings made anywhere, but my riding skill will not appreciate one bit until I put some training time into mastering them). In testing, I can have the greatest testing tools ever developed (and yeah, I think TestComplete, Selenium, twill, WatiN, Visual Studio and NUnit are some pretty sweet tools), but I will not be effective until I spend the required time getting the technique down. Otherwise, to torture my snowboard metaphor a bit more, I’m just a guy on an expensive rig heel sliding down a steep trail. I’m getting down, but it’s nowhere near as fun as if I use the full technique and skill required (and it’s a lot more tiring, too).

As I take more steps towards that wonderful reality that I hope to see which is meaningful test automation, I have to remind myself of my daughter this weekend, to get truly good at it and to cover meaningful ground, one must train on the basics, one must practice regularly, and one needs to start small and work up to bigger and better performance. It’s not fair to ask my kids to perform at the level of their dad who’s had more than 30 years of active experience on the snow. Likewise, it’s an unfair expectation to think that I’m going to be a master of tools I don’t spend enough time working with to master. It’s my hope in these coming months I will use this weekend’s reminder from my daughter to “focus on the basics” and make sure I have a good handle of those as I work my way forward… now if I can only find the coding and testing equivalent of “ride like a boxer”, I’ll be in good shape!

Using James Burke’s CONNECTIONS to Aid in Testing

There are places and times where revolutionary and paradigm changing discoveries can be made. Socrates and Aristotle changed paradigms, so did Newton, Einstein, and many others. The way we were able to look at things and the map that shaped those views was forever altered by these thinkers. In the everyday world, few of us get to have the ability to change paradigms. To understand how hard this is, it’s important to make the comparison with something that really did change a paradigm.

Here’s an example I remember reading about many years ago regarding breakthrough discoveries in Mathematics (J.J. O'Connor and E.F.Robertson wrote about this in 1997, the full URL can be seen at http://www-history.mcs.st-andrews.ac.uk/HistTopics/History_overview.html):

A challenge


If you think that mathematical discovery is easy then here is a challenge to make you think. Napier, Briggs and others introduced the world to logarithms nearly 400 years ago. These were used for 350 years as the main tool in arithmetical calculations. An amazing amount of effort was saved using logarithms, how could the heavy calculations necessary in the sciences ever have taken place without logs.


Then the world changed. The pocket calculator appeared. The logarithm remains an important mathematical function but its use in calculating has gone for ever.


Here is the challenge. What will replace the calculator? You might say that this is an unfair question. However let me remind you that Napier invented the basic concepts of a mechanical computer at the same time as logs. The basic ideas that will lead to the replacement of the pocket calculator are almost certainly around us.



We can think of faster calculators, smaller calculators, better calculators but I'm asking for something as different from the calculator as the calculator itself is from log tables. [...] Think about it and realize how difficult it was to invent non-euclidean geometries, groups, general relativity, set theory, .... .

I remember listening some time back to Dan Carlin’s “Hardcore History” podcast, and in particular, his interview with James Burke. For those not familiar with James Burke, he is an influential historian who developed and presented the "Connections" series on PBS and BBC back in the late 1970’s. His idea was to look at the notion that knowledge, and history, isn’t just a linear march from point A to point B, but that there were many cross sections and encounters and ways of influencing people that work its way into history. James makes the point that he could draw connections from Mozart to the helicopter in less than ten jumps, and only touch on music twice. It was James Burke’s Connections idea that has spawned something that I do in testing now that I also refer to as “Connections Based Testing”. 

The idea is not far removed from integration testing, but the method I use is to see how many different avenues and paths I can travel down without going back to the same section, and see how many “maps” I can create. By necessity, some of these maps will be very short, because there will be very little interaction with another unit, but often, there can be marathon stretches of code interactions that will take many, many steps. I find these options to be interesting test case stories, as often they will open up new avenues that I otherwise would not consider. This becomes especially important if an application integrates with other applications (say, Outlook, Word or Excel in the large application sense, or with Flash, JavaScript or Ruby in the web sphere).

I recently had an opportunity to see where an application I was working on interfaced with multiple moving parts, both within the application and calls to external applications, and determined that there was a triangle effect in place, that Part A called Part B, Part B called on and used elements of Part C, and then Part C referenced Part A again. It was in this triangle, if anything broke on any of the legs, the functionality would be broken all the way around, or would work much less effectively in the perception of the stakeholder (which many times equates to being “broken all the way around”). If we tested these items individually, we would not see the specific issues, but if we followed the connections, we would see the problem, and be able to determine what actually went wrong.

During the interview, James Burke confided that, as a young graduate student, he had met a professor who, in the footnote of a book, had made the point that the development of the stirrup in pre-medieval times had had a profound effect on shaping the history of Western Europe from the Fall of the Roman Empire to the Renaissance;  and he stepped through the connections showing how the stirrup allowed a soldier to stay armed in battle and really plow over the other guy in the fight, who then responded by putting armor onto himself, which caused the original warrior to put on heavier armor, which spread to an industry in producing super sturdy horses, who needed land to graze on, which was appropriated from the church and other places by men who came to be known as Dukes, and the process continued and developed to the point where huge horses carried superbly armored men into battle for city states that developed into nations. All connections, and James Burke like this idea, so when he approached the man who wrote the book to ask if he could use this idea about connections to explain historical phenomena, the professor responded with “I stole it, you steal it!” When Burke answered back incredulously that the professor had “stolen” this idea, the professor gave him a wry smile and said “my boy, you don’t think we are born with ideas, do you?”

The point is that we all borrow from each other, we “stand on the backs of giants” and we develop ideas from other people’s ideas. When that happens, give credit where credit is due, but try to see if you can take those ideas and carry them into different directions. I’m grateful to James Burke for "stealing" this professor’s idea, as it helped me develop and work with a “paradigm” of testing I hadn’t earlier considered. It’s by no means a new paradigm, but it was to me. How have you stood on the shoulders of giants to change your paradigm as to how you see and test things?

Applying a "Dread Pirate Roberts" Attitude to Testing

This is expanding a little on a post I made back in October of 2009 on my personal blog. I have been an advocate of the approach of considering yourself as an entrepreneurial consultant in whatever line of work you happen to be in, whether you actually are a consultant or work for a company and are on their payroll. As I was musing about what I’d want to call my consulting firm, I came upon the idea of “Dread Pirate Roberts Consulting”. Understand, I didn’t actually use that term, because Dread Pirate Roberts is a name closely associated with a beloved movie, but it was the idea behind it that has stuck with me.


For those who are fans of the movie "The Princess Bride" you will know this character. He was portrayed by Cary Elwes (the alter ego of "Wesley") and the foil for many of the characters to go up against. The parts of the movie I remember the best were his quips when he and Buttercup were walking through the Fire Swamp and he was describing how he became the Dread Pirate Roberts. I'll not spoil the whole setup for the perhaps two or three people out there who have not seen this movie, but there's a section where the "Dread Pirate Roberts" takes in Wesley by saying "well, I've never had a valet before, so let's see how this works, but be warned, I shall probably kill you in the morning" (paraphrased).


I've long thought on this quote and wondered "what would I do in my everyday life if I had that as a way to work and to live, the notion of 'work hard, work well, do what you can, I shall probably kill you in the morning'"? How would you do things differently? How would you approach problems? How would you approach projects?


This gave me a simple idea to play with in my work life, and something that allowed me to make a basic order of operations to maximize my effectiveness. I call these "the DPR Consulting Order of Operations" as relates to testing. the goal is to maximize effectiveness and make sure that I get in order the things I need to do because, hey, he'll probably kill me in the morning.


Management Directive: Let’s face it, when we are working for someone, as a consultant or a direct report, whatever work is most important to our stakeholders needs to be our primary focus. Note, that doesn’t mean “do whatever the stakeholder tells us to” or “go along with everything the stakeholder says”. It means that our primary responsibility is to find out what the single most important thing to our stakeholders is, and once we have done that, make sure that it gets done. If the number one thing is to “ship the product on time”, we owe it to our stakeholder to tell them everything that we know that could delay that possibility and offer what we could do to make sure that doesn’t happen. Notice I didn’t say “cut corners to meet the deadline”. When you are a consultant, you are paid to tell the truth about a situation, to the best of your ability, and help make sure that the project is a success. As a consultant, you have to be willing to be the bearer of both good news and bad news, and the directive may change. Be prepared for that first and foremost.


Bug Database: In my world, the bug database is the hive where most of my activity revolves. When I write and revise test cases, it comes from interacting with the bug database. When I want to find out where we actually stand in the cycle, I look to the bug database. When a project has a new build, or a new set of features, the bug database is my primary source of information, even more so than requirements documents, especially when those bugs come from requests in the field from customers; while requirements may say one thing, customers needs and demands may not jibe exactly with what has been put into the requirements document. In this case, the bug database trumps the requirements document, unless the stakeholders decide otherwise. My mode of operation is that I review all issues, determine if there is enough information to be meaningful to the developers, and likewise determine if the fixes have enough information to be meaningfully tested. If not, I have the choice of improvising and seeing if they have been fixed, or I can have the developers clarify what they have done to fix the issues and get specific thoughts on what to test. It is not an insult to have developers give clear guidance to what they have fixed and to make recommended use cases to test the fix. I cannot use that as my *only* method of validation, but I personally appreciate it when developers give details about a fix and what to look for, rather than to say “this has been fixed” and offer no other comments. As a tester, you should hold developers accountable for the changes they make and request guidance where necessary as to how to test what they have fixed (where possible).


Test Plan and Test Results Documentation: Many would put this first in their ordering. I put it third because I feel that it actually follows on from the information supplied from the first two. Please note that this ordering is never 100% absolute, but is a general flow of responsibility and action. Test Planning happens all the time, not just at neat little intervals. Test Documentation also incrementally changes over the course of a project; initial reports may be very informal, while project closeout will require a much more detailed and formal report of testing deliverables. All must be developed along the continuum. For me, my best test cases rarely come out early in a project, they are coaxed out in the intervals of builds and issues fixed and discovered while looking at those fixes. Going back and adding to test plans from this process is valuable and helps make testing documents and test cases stronger (it’s also the reason why I write almost all of my test plans in Excel rather than using a word publishing document or a Wiki. When I am ready to submit a completed test plan, *then* I put them in a more formal setting).


Test Scripts and Test Automation: While many would want to put this higher on the priority list, I feel it’s important to get the clear view of what you are testing first, and then work to automate things once you have a good understanding of what is supposed to happen. From there, test oracles can be established, data for applications can be thoroughly defined, and the tester can make comparisons to state which values should appear (if so, PASS) and which ones should not (else FAIL). I believe in using an unscripted approach at first, so as to get a feeling for whatever product you are working with. The scripts come later, and with that I mean the flow of operations, the user stories, all of the things necessary to interact with the application on a meaningful level with a high amount of certainty that you are covering what users would, and then some. The automation factors come later, and I will confess is often a late in the game item for me, or is something I focus on when I have clearly identified the expected behaviors of a variety of inputs.


Environment Maintenance and Update: The test lab, or test environments, have to be maintained, so any steps I can take to keep these up to date, with fall-back to previous states, and making sure that I have everything on tap to run with. To use my Dread Pirate Roberts paradigm, Environment Maintenance and Update is the equivalent of making sure a ship is ready to sail. Yeah, I know this stuff is really obvious, but this blog has to do with my own realities and what I have learned from them, so bear with me if I talk about some stuff that potentially has a huge “well, duh!” factor. To this end, I make sure that I have whatever tools are necessary to make sure machines can be deployed rapidly, set up as quickly as possible, and do not require a lot of hand-holding to get into a testable state. There’s no way to get rid of all the tedium, but certain tools such as MagicDisc, MagicISO, nLite and a collection of ready-to-go ISO images helps minimize the time required to set-up environments, and the judicious use of check-points allows a user to quickly get to the environments I need as quickly as I can.


BONUS Section: Read a Book a Week: Pick a non-fiction book, related to testing, or engineering, or software development, or marketing, or personal development… anything that will help you develop additional domain knowledge, and commit to reading a book every week. The time is not exact, but it’s a good goal to encourage people to get into some new ideas, challenge their current methodologies, and see if there is something that you can learn from in the vast world that’s out there. The answer, at least to me, has proven to be “you can learn from many sources, and many disciplines will help you sharpen your testing saw, even when the book in question has nothing to do with testing”… at least not at first. You will be surprised at how many books you can read on many different topics that will help hone your testing skills (I’ll be sharing some examples in the coming weeks :) ).


Here's wishing you all great days and great efforts, whatever they may be. May you not be killed in the morning... oh, and do your best to avoid the R.O.U.S.'s, too ;).

Lessons Learned from a "Testapalooza"

I know some of you are scratching your heads at the title, so let me explain. My last post dealt with the issues and realities of being what I call a “Lone Gunfighter” in a company, i.e. the single dedicated tester. Of course, when there is only one person in a company who is dedicated to testing, that leaves a whole lot of potential gap space. I have long ago come to the conclusion that it is impossible to test anything 100%, and that to put the product in the hands of people who are not testers per se, but who have experience with a product and can work with it, they are likely to find things that you have missed. What’s more, they are going to find things that become blindingly obvious once they have told you about them.

Thus, in my company, about twice a year, we hold what we have dubbed “Testapalooza” events. It’s basically a set amount of time (usually two hours) where we set up ten machines, get ten people from different groups in the company, and give them an opportunity to go wild on the application being tested. Our best participants have been our sales team and our customer support group members. Developers can participate, too, but only on projects they are not directly performing development work. I, however, am not allowed to test. I act as Proctor for these test parties, and I answer questions, offer clarification when necessary, but more times than not I utter something similar to “Oh, would you look at that? Yeah, that’s an issue, go ahead and write that up”, while under my breath saying “You have got to be kidding me… I missed that?!”

One could get the sense that Testapalooza’s could be seen as a form of punishment for the tester, an indictment on their skill (or lack thereof) in finding problems. I will admit that the first time we performed this exercise that was exactly how I felt. I saw the bug list created, and it wasn’t trivial. There were a lot of issues that I felt I had tested around and passed that, upon further review and inspection, showed areas I had missed, and often, areas I had totally not considered or even looked at. I thought for sure I was going to be shown the door after this, and I think I stayed up half the night afterwards wondering if I was going to get canned. The good news is that, no, I didn’t get canned, but I learned some valuable lessons from this exercise, and now I’m a fan of it. From fear and dread to acceptance and embrace? How do we get from there to here?

1. Every tester brings a different perspective: I realize this may be trite to say, but this cannot be over emphasized. No two people have the same view of the world, and no two people will approach tasks the same way. What is acceptable to one person will be downright annoying to someone else. One person’s common practices will not be someone else’s. Some people like to enter text a particular way, using the keyboard and every shortcut in the book, while others like to use the mouse and bounce around. Perspective and habits are important, and ones habits can have huge impact on results, even little things that might never rise to the conscious level (ah, but shouldn’t us testers be aware of those mannerisms and habits? Yes, we should, and as a result of these Testapalooza’s I am now more aware of them than I used to be).

2. Inattentional Blindness is real: I owe Cem Kaner for this phrase entering my lexicon. Cem showed a testing video a few years back highlighting this phenomenon. The most famous video of this can be seen that showed people playing a game with a basketball and the people passing the ball around. The video then asked if anyone saw anything out of the ordinary take place. About 50% of the respondents said that they did not see anything out of the ordinary, even when shown on playback that a man with an umbrella had walked into the picture or that a man in a gorilla suit had likewise walked into the picture. I’ve seen these videos, and I’ll confess, the first time I saw them, I missed the abnormal things, too, because I was so focused on counting how many times the ball was being passed. In testing, we have the potential to be thrown off by Inattentional Blindness, because we see things all the time and focus on key areas, but fail to see others, even when they are right in front of us. It’s only after they are pointed out that they are obvious.

3. There’s always a different order to doing a task: if you ask ten different people to do a job, but don’t tell them exactly what to do, it’s a good bet that no two people will do exactly the same things. The differences may be very subtle, but timing, order, duration, choice of terms, etc. will invariably happen. This is why, if you get a chance to get many people looking at a goal or a task, you will be much more likely to uncover issues because of the different methods people use. As a tester, you may look at something and say to yourself “now come on, I did that!” but on closer examination, your original method didn’t uncover the problem, but just a slight variation showed you a different way to do it. Of course, after seeing the slight variation, you then train yourself to check it in that manner, which when it is fixed, you keep it in mind to look at it with that calibration.

I’m happy to report that I have since come to appreciate and enjoy these opportunities, because I decided that it was way better to learn from them and improve our testing practices than it was to get frustrated and defensive about not being "the one" to find those issues. Yeah, I admit it, I can get defensive and frustrated when things that now appear obvious are missed. What’s important, though, is to note that these same issues didn’t just get past me, they also got past our developers and they also got past a number of other people. It took the combined forces of a number of people to confirm that, yes, indeed, there are more issues to be found, and more testing would of course be beneficial. At that point economics of scale come into play. Yes, these Testapalooza events are a great way to kick some adrenaline into testing efforts, and have a great chance of uncovering many additional issues, but having these individuals work on testing all the time like this would be incredibly expensive, since these people would not be doing their primary jobs (which likewise add benefit and revenue to the company). So I do my best to learn from them what I can, review my test plans, test cases, and methods of testing, look at what I can modify to incorporate the new information, and see if the next time, we have done better. It also tell everyone that quality is not just up to the test team, but it’s everyone’s job to make sure that we put the best product out that we can, and its way better for us to find these issues than having our customers find them.

So to all of my testing friends, if you don’t do Testapalooza’s, consider them. You just might be amazed at what your co-workers can help you find :).

Testing's "Lone Gunfighters"

Throughout my career, I have had the opportunity to work with various test teams, both as an individual contributor and as a team lead and test manager. During that time, and currently, I’ve also worked where I was the ONLY member of a test team. This can be both liberating and frightening. Liberating in the sense that you can do what you feel is appropriate, but frightening in the sense that there is no cover if things go wrong; if you mess up, it’s ALL YOU!!!

There are different levels of being what I refer to as a “Lone Gunfighter”. Sometimes you are the lone tester for a project, but you still have the resources of a test team to turn to. Sometime you’re the lone tester for a division in a given area, but you still have people you can communicate with in the testing department for your company. Sometimes, however, especially in smaller companies, you can be the “Lone Gunfighter” for the entire company. If you find yourself in this position, then this entry is especially for you.

When the situation of the “Lone Gunfighter” happens, testing takes on an interesting hue. Now, it’s all on you, and when you are the last line of defense, you really are all there is between the product getting out into the wild and bad things potentially happening. Sobering is a good word for this situation.

So what does an everyday software tester do when they are the Lone Gunfighter in their organization? From my years of being in this situation, here are my suggestions:

1.       Consider yourself a consultant for hire: This is a mindset that I feel is very helpful to the lone guns out there. Many times, we can feel like we have no direction, or that we are being barked at to get this or that accomplished, with little to no support from others. By putting yourself into the mode of consultant, you change the relationship. Now you are providing a service to a customer, and in this case, the development team and the ones purchasing or using the product are your customers. When the development team becomes a customer and your goal is to provide top notch service to that costumer, your entire mindset and focus changes.

2.       Get buy-in from your development team as to their expectations: In smaller companies, the development team may be only slightly larger than the test team, and therefore may be just as harried and under intense pressure to meet deadlines as you are. Make sure that you define your role as clearly as possible, and what you feel the expectations for your contribution should be, and let them say what they feel it should be as well. This agreement may be formal and in writing, or it may be something discussed in meetings or between team mates. Either way, get it out there so everyone knows the expectation and can work to meet or exceed it.

3.       Get over the us vs. them mentality: If you have no idea what this means, congratulations, you work at a company that “gets it”. If you are all too familiar with this mentality, make the first steps to change that dynamic. Testing and development are allies, they both have the same goal, to ship a product that has high quality and that will meet the needs of its customers. No other attitude is going to help that other than development and testing working together to help solve problems. That means any adversarial attitudes should be stopped, and often the adversarial issues come from testing, whether we intend them to or not. Look at the way you write issue reports, or the way you speak in meetings. If you find that you are pointing fingers at developers or if you reread bugs written and you see attitude and innuendo in the writing, vow to change that.

4.       Realize that Test Tool decisions may mostly be yours: most Lone Gunfighters are in this position because the approach to quality or testing is relatively new in the life of the product or the company (not always, but that tends to be the case). As such, it’s very unlikely that a structured mechanism for testing or using test tools is in place. This provides both challenges and a great deal of freedom at the same time. Challenges in that a small organization is not likely to spend a lot of money on a proprietary test tool (though some might, if you are exceptional in using it and can make a strong business case as to why that tools might be the way to go). They might, however, be very open to using open-source tools. More often than not, though, there may be no tools at all in place, and this may be a great opportunity to explore and implement tools you already know or, barring that, have a hand in recommending what the group will use going forward.

5.       Accept that you will not be an expert in everything: You will have knowledge gaps, and at times those gaps will be huge, but you have to make a clear assessment of your strengths and weaknesses and put them out there. Yes, make them known, the strengths and the weaknesses. Also, make a game plan to overcome those weaknesses where possible and telegraph the fact that you are working to close those gaps.

6.       You won’t have time to do everything: This is just a fact when you are alone in your efforts. Testing of multiple products, setting up multiple priorities, writing multiple test plans, working on automation, creating new test environments and keeping them up to date, all of this has to be juggled and maintained. It’s not possible to do it all, or more accurately, it's not possible to give all the same priority at the same time, but it is possible to have your stakeholders know what you can do and when. My simple and old school method of this is a white board on my wall that faces out to everyone in the hall, along with what have become known as my “Morning Declaration” and my “Evening Update”. The “Morning Declaration” is where I let my stakeholders know what I have planned for that day, and what spheres those plans occupy (testing, documentation, bug database maintenance, environment setup, automation, etc.).  I believe in “thrashing early”; get my plans out to everyone as to what I am doing and give them a chance to have their say. Once that’s done, barring a major catastrophe, I stick to the plan.

7.       Find an outside mentor: When you are Lone Gunfighter, you may be able to find someone who can help you in the development group, but often they may have limitations as well (especially around testing questions and issues). Reach out to other testers you have worked with and ask questions (within reason, these people have jobs, too J ). I still have mentoring relationships with friends who were instrumental in my development over the past 20 years, and I also occasionally get asked questions as to something I have experience in. I believe it’s important to have a mentor and to be a mentor, so look for opportunities where this can be utilized. Fortunately, in this day and age, sites like LinkedIn, uTest and others have communities where testers can reach out to other testers and get some advice on a topic or area where they are not necessarily an expert (and when you are a Lone Gunfighter, that situation happens a LOT of the time). What’s very cool about our community is that we tend to help one another; it’s rare that you will get snubbed for asking a question (it helps of course to do your own homework first, and narrow down to specific questions where possible, but if you’ve done your part up front, most testers are more than willing to help a fellow tester out).

To my fellow “Lone Gunfighters”, I feel for you, and I understand the occasional loneliness of your role. Nevertheless, realize that you are a critical element in the success or your endeavor, and in many ways, you have the potential to become “the indispensible one” where you work, as well as the one that the team ultimately grows around when the situation and growth of the company calls for it (and if you actually want to go that route; some Lone Gunfighters deliberately make careers out of being exactly that). For those who are Lone Gunfighters, what have you found to be your biggest challenges, and what have been your best ways to deal with them?

Test Training? What’s That?

To be fair, the above statement isn’t as common as it used to be. There are now many options regarding testing training and how to get it. However, up until just a few years ago, being a tester and getting actual training related to testing was not an easy endeavor. Training for programming? Yes. Training for systems administration? Definitely. Training for engineering principles? Just look at MIT or Caltech or any number of engineering schools. Still, you would have been hard pressed to find a school that had an actual course of study related to software testing (and for that matter, just finding a single course was not a wide spread available option).

Show of hands, how many testers had any genuinely formal training in how to do their job short of perhaps a quick orientation from a test team lead or test manager? How many people actually participated in training beyond this introductory level? If my experience is indicative of the majority of testers, I’m willing to bet that number is probably very low. I have participated in product training and application level training, but testing training was always something I had to cobble together on my own from books or from web sites and discussion forums. Even when it came to the one testing training that I can point to where I was flown out to attend a week-long training event, it was to specifically learn how to use a tool, not how to effectively test with that tool. That I had to figure out on my own (and I’m still in the process of doing exactly that, it seems :) ).

Why has there been such a disparity? I can’t speak for every organization, but for the ones that I have been part of, to put it simply, training testers was not high on the priority list of things to do. I’m not sure if it stemmed from the idea that testers already knew everything they needed to know (common perception in many companies) or that the work they needed to do didn’t really require additional training (again, common perception). Many of the testers I’ve spoken to over the years had little to no formal training in testing, and of that grouping, there was a high percentage that had not read a testing book or, if they had, hadn’t read one in several years. Truth be told, I fit into that category for a long time (the only book I owned was “Testing Computer Software” by Kaner, Falk and Nguyen, and seriously, I figured that was all I’d ever need).

It turns out that there has been testing training offered for many years from various organizations, and once you learn about it and plug into it, learning begets more learning. There are many posts about this on the web and many companies you can talk to if you want to pursue training. Since I’m not affiliated with any of them, I’m not going to mention any by name, but if you do a search for “Software Testing Training”, you will find upwards of 30,000 links, many of them pointing to “for pay” organizations. For the record, I cannot vouch for the effectiveness of any of these courses at this time simply because I haven’t dropped the coin yet to pay for them, but I have seen several companies sample offerings and in my opinion the ones I have seen range from mediocre to exceptional.

That’s all fine and good, but what about those of us who are working for companies that have limited training budgets or, gasp, have no training budgets? Or what about those of us who are contracting and don’t have the ability to pay for a $300/day course or a $1500/week option to be trained? The great news is that there is a ton of information that you can get very cheaply or even for free on the Internet today. Here are some examples of cheap and free test training options:

Testing Blogs: If you look to the side of these posts, you will notice that I have a roll of a number of test blogs. There are hundreds, but these are the ones I keep returning to again and again. I return to them because they challenge the way that I think about things, or they provide me with solid information and ideas to explore. There are hundreds more out there, and to be totally honest, I’d love to have TESTHEAD fill that role for people as well (it’s going to take awhile to develop the credibility so that it will be worth that to someone, but hey, I aim to try :) ).

Testing Magazines and Communities: There are several magazines and groups that publish information that is readily available to everyone, and while they still require a subscription fee to get everything they offer, the amount of information they offer to the general public for free is substantial. Some of my personal favorites are Software Test and Performance magazine and their online collaboration site and Better Software magazine and the folks at Sticky Minds.

Webinars and Online Training Sessions: both STP and Better software listed above host a series of webinars that cover different areas of software testing on a regular basis. Most of the sessions are free and are open to the public, and many of the sessions are “tool agnostic”, meaning that they talk about principles and practices that can be used with any of the common test tools, or applied to home grown solutions. There are also a number of webinars that are sponsored by tool vendors that provide an opportunity to see the tools in action and show how they can answer certain issues in testing. Note: just because a tool vendor sponsors a webinar, it does not necessarily mean that the entire session is a sales pitch for the product. While that may be their desired goal, there is much that can be learned from them that doesn’t require dedicating yourself to purchasing their tool or using their product.

Books: Yes, those antiquated pieces of paper held together by some binding and between a hardcover or paperback can still really be tremendously helpful. I’ve already mentioned “Testing Computer Software” by Kaner, Falk and Nguyen, but there are many others worth reading and that will give you some great training that can carry over into many different disciplines. William Perry’s “Effective Methods of Software Testing” is a desk reference in the literal sense of the word, and it provides a lot of information for many different approaches to testing (more than most testers will ever use in their careers, in fact). I would also like to add to this list a few additional titles that I think any tester would be well served to also read (note: some of these books are not testing books, but they cover areas that I think any tester would be well served to read and learn from):

• “Linchpin” by Seth Godin

• “Secrets of a Buccaneer Scholar” by James Bach

• “Software Testing: Fundamental Principles and Essential Knowledge” by James McCaffrey

• “Software Test Automation” by Mark Fewster and Dorothy Graham

• “Surviving the Top Ten Challenges of Software Testing” by William Perry and Randy Rice

I plan to do more extensive reviews on each of these books in the future, so stay tuned for those.

Wikipedia’s Software Testing Portal: Wikipedia is an example of, to borrow from Seth Godin, “the law of the Mechanical Turk in action”. The law of the Mechanical Turk refers to the ability to take something that would be too huge to develop and maintain for one person or a small staff, but becomes easy to develop and maintain when millions of people have access to it and provide a small part. The Wikipedia software testing portal is an example of where this vast resource of people and small contributions comes together to make for a very large repository of information related to testing. Note: a phrase I famously use among friends and colleagues is “Caveat Lektor Wiki”, and this is no exception. Using Wikipedia as a sole resource for anything is never a good idea, but to get started and develop some basic ideas and understanding, it’s a great tool, and again, will provide many jumping off points if you wish to explore them.

So, how about you? How have you gone about receiving training in software testing? For those who have specifically taken the extended course offerings through conferences and training organizations, how do you feel we are being served? As I said, I personally have not committed to a full training course through any of these companies, but many of the offerings have piqued my interest. What are your favorite web training resources, and how have they helped you? What would you like to see in the future regarding testing training, and who would you like to see offering it? One thing’s for sure, there’s no better time than now to finally get some clarity on this topic. The knowledge is out there, and whether it be formal or self directed, it’s entirely possible to get the training necessary to develop, survive and thrive in the testing world. If we don’t avail ourselves of the opportunities out there, we have no one to blame but ourselves.

Can You Really Test Anything?

A couple of days ago, I commented on the idea that there are those who see testing as an "easy" profession, and that "anyone can do testing". I disagreed with this statement and said that, actually, it was quite the opposite, that it takes a certain type of temperament and skill set to be good at testing.

There's another piece to this, and it deserves to be mentioned as well... the idea that testers can test anything. That's partially true, but just as I said that to be truly good at testing, not just anyone can do it, likewise, it's very possible to be a superstar tester in one endeavor and struggle in another. I've had these experiences over the years, and I know how it feels to be flush with victory in one sphere, only to be slapped hard with the realization in your next endeavor that "you aren't as good as you think you are".

I came into testing like many people did, from a sideways trajectory. Truth be told, I did not go to school and say to myself "you know what, I'm going to be a tester when I finish school". As a matter of fact, outside of using a tiny old school Macintosh to put together fliers for my band, I barely had any interaction with computers at all. I was working as a housekeeper and a part time student by day and as a musician by night when, in 1991, I decided I needed something more stable to help pay the bills as a musician. I took a temporary job to help pay the rent while waiting for super-stardom. Well, super-stardom never came, but that temp job grew into a ten year long career, and one that introduced me to the world of software testing and the people that did it.That experience took place at what was then a little, barely known company called Cisco Systems.

As I interacted with the testers, I learned about things related to networking interfaces, and how to configure switches and routers, and how to build networks. From there, I learned what it took to get data from point A to point B and what I could do to cause problems in those transactions, but see if the systems would recover. In short, I discovered that testing was fun, but challenging to be effective. I had some great help along the way and worked with some great people who taught me many things.

Several years into this process, I decided I wanted to go into the Network Management side of things and test Network Management applications. I figured all of my experience with routers and switches would make this a breeze... I was very much mistaken. This would be one of the first times I realized that my skills were not entirely transferable.

Lesson #1: Not all testing is created equal, and just because you know how to do some things very well, it's no guarantee you will do well with a similar product or even in a similar area.

Lesson #2: I had worked with a team of people when I was testing routers and switches. When I moved into testing Network Management software, I was a lone gun. I learned quickly that when you stand alone, there's no one else to fill in the gaps, so you better figure out what you don't know quickly, and if you can't, get onto something else where you can be effective.

Later on in my career, I decided to leave Cisco Systems and try some other options. For two years I made a stab at being an Application Engineer at two separate companies, but in both cases, I realized that my testing skills, while definitely helpful, were very low on the totem pole of what I was expected to do (actually, I think the only more nebulous job description beyond being a tester is to be an Application Engineer; it seems that whenever I asked people what I was supposed to do, each person had a different answer). One of my stints was at a company that made specialized interface devices that worked with human touch. As such, the company was focused around hardware development with a software interface to communicate with the system. Hey, I'd worked with routers and switches, these wouldn't be that hard to figure out, right? Ha! I soon learned that not having a strong background in physics put me at a distinct disadvantage in the group I was working in, and because of that, I really struggled to test these products effectively or to even understand what I was testing (though I have to confess that learning about the human body model and doing ESD testing was a lot of fun in a twisted way).

Lesson #3: Hardware is not software, and different hardware skills are not necessarily interchangeable. Each time you find yourself working with a different product, you run the chance that you are starting off at almost ground level.

Lesson #4: Actually, you don't really start at ground level, but you do end up using what Dr Rebeccah Reinstein refers to as the "spiral staircase" phenomenon, in which you come back to the same place, but a little bit higher each time.

While I did learn quite a bit at this job, it became clear that I was not a good long term fit in this position, and so I decided to try something else. I went back to school and while I was in school, I went to work for a game publisher. I figured, hey, I like games, and I have 12 years of testing experience, this will be a piece of cake! It took me about five days to realize that, no this was a paradigm with its own rules, its own methods, and that there was a whole vocabulary to testing that was totally different than anything I'd ever done before. My general testing skills helped quite a bit, but I was definitely outclassed by those who had done this for years, and it showed. I did pick up a lot of great tips and methods and learned to see things in a different way and became a pretty good game tester, but I'll be the first to state that my experience in the game industry was a sobering one; my 12 years of general skill was perhaps 25% helpful. The other 75% was domain specific, and I had to learn that from the ground up just like everyone else.

Lesson #5: General skill is important, and knowing how to tap it is helpful, but it will only carry you so far. Primary domain knowledge is the deciding factor between who is adequate and who is genuinely good. The good news is that, if you have the fundamental knowledge, the domain knowledge can be developed in time, provided you are willing to put in the effort to learn it (and yes, when it comes to games, there is a lot of domain specific knowledge to learn).

Lesson #6: People think it is fun to test games, and it is, to a point. However, people hoping that their enthusiasm to play games will make them great game testers tend to get a bit of a rude awakening. It's work, just like any other, and to be successful, you have to have more than just a desire to play, you have to actually learn the ropes of what it takes to be effective in finding the problem spots, just like any other application.

Today, I work in a unique capacity. Again, I'm a lone gun, and again, I work with a product that has its own special quirks, and it took me some time to learn the ropes of what it takes to be an effective tester in this sphere (and at times, I question just how effective I actually am). Believe me, I have my good days and my bad days, but after almost 20 years, I have had enough turns on the Spiral Staircase to see that, yes, I can possibly test just about anything now, but to be effective, I can't just drop in and expect to hit regular home runs (that's very rare, actually, unless you are doing something very similar to what you have always done). I have learned, however, that there are many skills that I have picked up over the years that I can cobble together to help me test any number of things at a fundamental level, while developing the knowledge to go deeper as I get the domain specific knowledge necessary to be effective, and knowing that that domain specific knowledge will be different just about everywhere a tester goes. It certainly makes work interesting.

My Quality Assurance Toolkit

Just as a carpenter would never dream of going to a job without his tools, we also need to decide what our tools are and how we can best utilize them. Different jobs require different tools. Some places are closed shops and only use proprietary tools or things developed in house. Others are more flexible and encourage testers to get out there and try new things.

Here is a sampling of my toolkit, and I will expand on each of these at a later date.

1. Test Plan Templates: Not everything is going to be tested the same way all the time, and you may likely never use the same exact test plan twice, but almost all test plans share things in common, and having a few ready-made templates to use for certain circumstances can cut down on the time needed to write a first pass test plan. I prefer using either Excel or a wiki tool to write test plans due to their speed and ready availability to others to review and make comments. Excel has another benefit that dovetails into the second part of my toolkit. It's easy to chop up test plan pieces to integrate into a "test dashboard".

2. A Test Dashboard: Again, the implementation will vary depending on where you are at and what you are doing, but in most cases, a simple dashboard will give you a heads up in how to communicate what you are doing to others in your organization. Dashboards can be elaborate or they can be simple. What matters is that you are communicating the relevant information in them and that your data tells a story about what you are seeing. I personally prefer a low tech method for dashboarding, which is to use an Excel spreadsheet model that I was turned onto a few years ago. It aggregates a number of items from manual test cases and scripts and then lets the user compile all of that data into a simple view. The other benefit is that this is a way that you can get rapid reporting and shows what you are doing while you develop a more sophisticated method later on.

3. Cygwin: This is a personal choice, but one that I do not like to go without. For many years I worked in a primarily UNIX shop, and I grew very accustomed to the methods and approaches to scripting in a UNIX environment. While I enjoy using UNIX and Linux, and keep some machines around to play with, there have been times where I have worked in purely Windows shops, and while Windows has a number of useful tools, there’s something to be said about the quick and direct elegance that UNIX pipelining of utilities provides, so cygwin is my simple compromise. If you have no familiarity with UNIX, then these tools may not be much of a help to you, but if you have worked with UNIX and wished many times for the ability to use awk, or like the direct scripting interface of the bash shell, or just find yourself way more effective editing text files using vi, cygwin bridges the gap very well.

4. A Scripting and a Development Language: Actually, there may be many languages you’ll need to know to be truly effective, but even if you only know one scripting language and one programming language really well, that can stand you in very good stead (and also give you a leg up on working with other languages). For me personally, I’m most familiar with C on the coding side. The shop I work in currently does most of its new work in C#, so there are a lot of familiar aspects between the two. In the scripting world, Perl is the language I am most familiar with (while I used to do a lot with Tcl/Tk, it’s been a decade since I used it in any meaningful sense). Still, having a background in Perl helps me to see common patterns in other scripting languages, and then just looking at the details helps me to fill in the blanks where needed. Again, getting up and running with scripting isn’t hard, but getting proficient with any language (programming or scripting) takes time and effort to get really good.

5. An Active Library Card: Some may scratch their heads at this one, but I’ll explain why I consider this to be important. The Peninsula Library System in the county that I live has 34 interconnected public and community college libraries; the ability to get access to books on just about any discipline fairly quickly is very valuable. However, the true gem in my library card is the subscription that it provides to Safari Books Online. There are hundreds of books related to software testing alone, and thousands of books related to programming, software engineering and any other number of topics, all available online and with the credentials provided by my library card.

6. Some Open Source Test Tools: What you use will vary from where you work and what your focus is, but I will recommend highly that developing some familiarity with an open source tool or a number of open source tools will give you and edge and provide you the ability to test things “right now”. These tools can range anywhere from basic macro recording tools for applications to sophisticated web based capture/playback tools rivaling features offered by the professional offerings. What’s more, most of these tools can be downloaded and used with little or no cost to you or your organization, and the skills you develop using these tools will many times translate over to commercial tools, too. Examples I currently use are twill, selenium and watir, but there are hundreds if not thousands of open source offerings to choose from.

7. Some Basic Reference Books:“Testing Computer Software”, written by Cem Kaner, Jack Falk and Hung Quoc Nguyen. I’ve had this book for well over a decade and still refer to it often. Another title I’d definitely recommend is William Perry’s “Effective Methods of Software Testing”. These two books together will provide a tester with lots of good practices and excellent ideas to apply to testing and help develop a lot of great skills. In addition, I would recommend getting books that relate to the business of the product you are testing. Many times we can be well versed in testing skills but without knowing the lay of the land of the particular business that we are dealing with, we will be much less effective than we could be (currently I work with software that helps with Case Management for Immigration Attorney’s. Due to this, my primary direct domain knowledge book of choice is “U.S. Immigration Made Easy” by Ilona Bray). Having worked in test positions related to virtual machines, networking equipment, capacitance touch devices and video games, I have owned and read books that are relevant to those spheres as well. Having a few titles to read and help you understand the domain that your software product is meant to work is the important part.

8. A Virtualization Library: While not every testing role or environment will be suited to being tested in a virtual environment, many applications can be tested this way, and having the ability to build and update environments on the fly can be a huge benefit. There are pluses to having commercial grade tools like Microsoft Hyper-V, but even without that level, users can create virtual machines easily and for free with tools like VMware Player, VMware Server and Virtual PC. Also, creating test environments with demo versions of products is a great way to test environments without having to purchase entire suites of operating systems and applications (the 120 to 180 day Evaluation copies are perfect for test environments, since the likelihood of an environment being around that long before needing to be reinstalled is pretty low, and if it will be around that long, probably justifies purchasing a license).

Is this *everything*? Not even close, but it’s a good representation of tools and techniques you can bring into any testing project, and be able to hit the ground running fairly quickly. Also, there’s no way to make it so that there is a simple “one size fits all” toolkit for every purpose, but the good news is that, at least with software, techniques are malleable and can be adapted fairly easily. At the end of the day, though, you will be the one that needs to create your own map and approach things your own way. These tools have helped me, perhaps they can help you as well. In the future, I plan to talk to each of these areas a little more in depth, but I’d also like to hear what tools you consider important in your testers toolbox.

Anybody Can Do It, Right?!

In a discussion I’ve been part of, an interesting question was asked. Can anyone become a software tester? Or more to the point, is Testing something everyone or anyone can do? The replies back were varied and interesting, and quite a few were rather humorous as well, but it brought out some interesting thoughts and ideas. First and foremost, those of us participating agreed that Quality Assurance is not “easy” to do. More precisely, it’s not easy to do if it is to be done well and consistently.


My thoughts from this exchange started to wander towards why this is a prevalent attitude. Do we get a bad rap in Q.A.? Why do so many look down their noses at “Testers” or "Quality Assurance Engineers” or "Test Analysts” or "[fill in the blank trendy job title]"? I don’t have all the answers to this, but I do have a few ideas. One of the biggest, I think, is the fact that our contribution can vary wildly depending on what it is that we actually do. In the software industry, developers get a lot of recognition for an obvious reason; they produce a shippable product that is tangible and people can use it and say “hey, this is great” or “wow, this isn’t good”. If you are someone who works with automation and creating test scripts, there is a little bit of this “shipping” phenomenon in that you have actual tangible items to point to (test scripts, test reports, etc.). For many of the things that testers do, there really isn’t a tangible product to point to and say “see, I made this”.


This works against us in many ways, because without a tangible physical product to show, we can easily be seen as either a non-entity at worst or a less than well understood part of the business puzzle at best. I want to argue that we do provide something of value, and that it is possible to demonstrate that there is a way that we show our value. It isn’t that we are the barrier to bugs, or that we guarantee a products quality (many testers have the idea that that is their primary responsibility), but that we ultimately provide a story. Yes, a story!


In my world view, testers have more in common with beat journalists and investigative reporters than any other job I can think of. We also have a lot in common with detectives. We are required to investigate a product, look for its flaws, and report them in a compelling way. When we say that “anyone can be a journalist” because anyone can write a few paragraphs, we tend to laugh, because we realize that to be a good journalist requires a lot more than just writing a few paragraphs. There has to be deep knowledge on the subject, or at least enough knowledge to know what to go after to expose a story. Journalists interview, they follow leads, they get both supporting and opposing viewpoints, and at the end of the day, they make a story that moves people to consider what they have discovered and add that to their world view.


Just as it would be an injustice to say that a great journalist is a commodity position, because anyone can do it, being a great tester is likewise just as challenging, but the reason we don’t get the same “press” as journalists is because we don’t actively tell a compelling story. Well, I believe the really good testers do. I’ve decided I want to be seen as one of those really good testers, so I’m going to do all I can to write compelling stories in the world in which I test.

Welcome to TESTHEAD

OK, why the need for a blog like this? Well, truth be told, I don’t know that there really is a “need” for this blog, but I’m doing this as a challenge to myself. I consider this an opportunity to “give back” to a community that has helped me over the course of many years, as I have been the beneficiary of many great insights and learned a lot from a number of people and sources over nearly two decades.

First off, professionally, I am a Tester. It's been what I've done in one way, shape or form for most of my career. As such, I am strangely drawn to the fine art of "breaking things on purpose" and then trying to find ways to improve the process so that they do not break again.

Being a Tester requires a bit of many disciplines. Saying "I like to test things" really isn't enough. A good Tester needs to have some understanding of the Software Development Cycle. This means that, to really be good at what they do, they need to "embrace the code", and I'll be the first to say I've had my fair share of ups and downs with that. They also need to have some skills with troubleshooting systems and finding solutions to issues. They need to be able to communicate to a broad group of people as to what they are doing, and ultimately, they need to be a part of the solution to the issue whenever possible. It's in the spirit of those areas I hope to contribute something here.

Most of all, this will be a site where I share my own experiences, both good and bad, and what I've learned from them. Expect there to be talk about tools, both proprietary and open source. Expect some talk about test case design (and how I so hate to do it at times). Expect to hear me vent about some frustrations at times, because like all people, I have my share of frustrations when things don't seem to work correctly or go the way that I planned them to. Expect me to share ideas on testing that don't divulge too much of what I do at my day job... much as I find what I do interesting, chances are there's not much anyone who is not in my particular niche market (software applications for the Legal industry) will be able to use outside of that area, but if I come across a cool concept or a neat way to do something, I'll definitely put a more generic example of it here. Most of all, expect to get a real person's perspective on these things and an attempt to communicate them in plain English, whenever I possibly can.