Learn When to Automate and When Not To: 99 Ways Workshop #5

The Software Testing Club recently put out an eBook called "99 Things You Can Do to Become a Better Tester". Some of them are really general and vague. Some of them are remarkably specific.

My goal for the next few weeks is to take the "99 Things" book and see if I can put my own personal spin on each of them, and make a personal workshop out of each of the suggestions.

 
Suggestion #5: Learn When to Automate and When Not To -- Christopher George

Automation… I figured we'd get to this topic soon enough.

I'm going to say it, and if this frustrates anyone, I apologize. If you want to become good at software test automation, you will have to become good at software programming. That's it. Where you do that programming is largely left up to need, purpose and application, but there's no way around it. Good automation requires good programming skills, and consistent use of them. Abstraction frameworks help some, and may ease the way in for some testers to help in the automation process, but those frameworks will only carry you so far. If they break, need service, or otherwise don't meet the actual need, then what will you do? The likelihood is that you will have to grab a programmer and have them help you fill in the blanks. There is nothing wrong with that, but it's a reality too few people understand, and too few tool vendors ever mention.

Now, having gotten that little downer out of the way... what I really want to do is reset the discussion at this stage. For our purposes, let's drop the phrase "automation" because it's too broad a description and it covers way too much to be a practical term for this workshop. Instead, let's bend the vocabulary a bit and use a different term… computer aided testing. Every time we do anything with a computer system, we are using computer aided testing. Any time we determine there's a pattern that we can repeat and, by bundling the steps together and running them in sequence, we can either save time or prevent human error, at its most elementary level, that is automation, and that is something anyone can start doing.

Workshop #5: Focusing on Patterns of Repetition

If you are working with a Linux/UNIX based system, this is something that you are probably already somewhat familiar with. The whole point of the UNIX paradigm is that utilities take work from one process and direct them to another process. This pipelining, redirecting, copying, sorting and trimming is at the core of most one-off shell scripts that are written. I've written one-off scripts for many years, but didn't stop to really consider what I was doing until relatively recently. Many software testers have similar steps that need to be performed. We start environments, we get them to a particular state, we pose questions that we hope will yield certain answers and if we see what we expect to see, we break down the environment or just exit the shell.

Chances are, many of the steps that are performed to do this are the same every time that you do them. If you can identify those steps, and the sequence those steps are performed, then you have identified a critical area where "computer aided testing" can be of benefit to you. Likewise, if at the end of a test run, you have a large amount of data that you need to go through, parse and cut out key areas to look at more closely, that is also another area where "computer aided testing" can be exceptionally helpful.

These areas are not glamorous. They are not what we traditionally consider to be "automation". They certainly aren't what we normally look at as being "automated testing". Still,  everyone has to start somewhere, and if you need to automate anything, I would strongly suggest automating those steps that you know will always, or at least a great percentage of the time, act the same way.

Another area that frequently comes into play is checking input fields. Much of the time, we know in advance what values we want to use, or a representation of what values we want to use. Filling in those fields manually is murder, but there are small tools that you can use that can ease this process considerably. On Mac, I use a tool called aText. On PC, a longtime standby for this purpose is Texter. Both work in similar ways; they allow you to create shortcut keys so that you can input text in predictable order. Both tools support tabbing, so that you can fill in forms directly. Again, this isn't classic hands off automation, but they streamline and simplify many time intensive steps, and that's often a very good thing.

Many people want automate their browser interactions, and for that purpose, using direct tools like Selenium IDE is a "gateway drug" to test automation. While I think Selenium IDE has a limited benefit, it can be helpful to do certain things, and it's a good way to get started with repeatable testing at the UI level. The disadvantages are also many, and there are plenty of blog posts out there that explain exactly why.

Most important, though, is taking the time to stop and think about what the actual problem is. As I wrote on the SmartBear Software Quality Matters blog, when we focus too much attention on a tool (or a need to automate) we often miss the obvious point, which is "what is the problem that we want to solve?" A tester might want to create a data driven test that walks through and adds 100 users to a system using the front end form. If the point of the test is making sure that the form handles a variety of input parameters effectively, then yes, a front end, data driven test might make sense. If the goal is to populate the database with 100 entries, then a much more effective approach (and likely less error prone) would be to create a shell script that inserts records to your test machine's SQL database (or the appropriate nomenclature for your system of choice).

Bottom Line:

For anyone looking to take their first steps into automation, start small, but start with what will give you the biggest immediate benefit. Make it a habit to become "more lazy". Find the tools that will allow you to repeat the repeatable, fill the fillable, and click the clickable. Once you get a taste of how to do that, then you may find you have an appetite to get deeper into more traditional automation frameworks and approaches. As you make that transition, you will need to boost your programming chops to meet the skill needs, but as you do that, you will also be getting better at recognizing real duplication patterns, and the more familiar you are with what those patterns are, the more effective you will be at making automation tests that will serve your needs.