Automation in Testing - Live at #AgileTD

Richard Bradshaw is a name a few of you out there are probably familiar with. Test Automation and Richard tend to go hand in hand, and Richard opens his talk saying that that might be a bit of a problem, and that he wanted to break free of that way of thinking.

In this talk, Richard is keen to look at the idea that "checks" are more accurate a phrase as in regards to automation. The bigger question to ask is "what does it mean when we perform test automation"? Are we "automating tests", or are we "automating checks"? What does the computer actually do for us? Computers are fast, accurate and stupid. Humans are slow, faulty and brilliant. Together, we can do magnificent things, but we are best suited to doing the things we are best adapted to. Automating is important, it needs to be done, and in many cases, it is a real lifesaver for a harried and overworked test team.

The key point to the whole "testing vs. checking" is that there is a need to use our brains to cover more and more areas. We can't remember it all, and frankly, we don't want to. There are so many steps that we have to keep track of that having a system that can keep everything running in sequence is wonderful. In these cases, I love having automation. It's especially helpful when you know that very step as defined will run the same way every time. The more that I interact with applications and I try to find things that are known issues, the more I find that I end up with a collection of terms and phrases. Each of those terms and phrases are areas I can search for next time. More to that point, commands I run over and over again, save a couple of variable changes, are wonderful areas for creating miniature tool chains (they can be a simple as a single line to parse a log file, or as large as an entire build sequence with a variety of possible conditions.

There are a lot of tools and programming environments out there, and many testers feel frustrated that they don't know how to use these tools. Yet these same "non technical" people do shell manipulations all day long, and many of them are saving those commands in executable files so they can save some keystrokes. These people are the unsung automators. Yes, they are doing automation. They just don't give themselves credit for it because they feel their scripts are not "sophisticated" enough. This is totally OK. This is actually a core part of the journey, and these early steps are key to getting the chops necessary to do more advanced automation.

We have to ask ourselves, why do we test? It's not to run the same things over and over again. Instead, it's to get new information about our product. Automation can help us find new things, but in many cases, it sequences events we know well, and looks for an event we expect to see. In these cases, green tests provide some comfort, but they don't tell us anything genuinely new. Automation will rarely lead us into new vistas, but exploratory testing certainly can. Every initial test we run is exploratory in nature, and as we get a better feel for what we expect to see, then we have space to create automation to cover the steps we know for the paths we have determined are important.