Walking Through the Land Mines

So many of the really interesting times I’ve had in testing have been to look at areas that no one considers or that have been around for years. There’s a sense that, even though the area has been combed multiple times, that there’s an issue (or a bunch of issues) that someone has missed. A phrase I like to use when describing a walk through old application details is "Walking Through the Land Mines". Depending on where you are in a project, you may find that you have different feelings about taking this particular journey.

For me, early in a project, it is fun to walk in the odd areas, because triggering a land mine can be fairly easy, and the repercussions will often be laughed about, quickly fixed and then we all move on to other things. However, when you get down to the wire, and you are at the end of a project, a walk through a mine field will not get you the same results. Over the years, I have had reactions ranging from suggestions to look at other areas to absolute admonitions of “do not consider that area part of the scope of testing for any reasons” and that “any issues found will not be fixed”. Translation: enter into the mine field at your own peril, and don’t expect to be rescued if you step on something.

The reason we look at land mines as a metaphor for software issues is the fact that land mines do tremendous damage in the heat of a conflict, but the real tragedies of land mines are frequently what happens years and even decades after a conflict. A fascinating book by Donovan Webster called “Aftermath” describes the conditions of battlefields after wars have ceased, and the job of “sappers” to remove land mines after battles have been fought, sometimes decades after the fact. To borrow a little from the Amazon.com review… “each year more than 5 million new land mines are laid, and only 100,000 are cleared; a new mine costs $3, but removing one costs between $200 and $1,000”. Does that figure ring familiar with any testers out there? Doesn’t that figure sound strangely similar to the notion of finding issues early costing $1 to fix, but finding them in the field costing as much as $1000 to fix?

So what’s a tester to do? Do we honestly just consider some areas of code off limits? Do we continue in the false security that code that hasn’t caused a problem in years should just not be poked too much because we might set off a chain reaction we might regret? While I understand the desire to leave well enough alone, honestly, I think we do a disservice when we do that. While we may not have the power or authority to fix everything, we at least owe our stakeholders (in this case, the development team) an honest exploration of all areas. Yes, that means we need to take a walk through the land-mines, and do so in a way that will potentially raise some scary issues. In this case, though, I am not outright advocating a "fix it or else" approach (i.e. when I do this walk, I go it alone, and set up my environments in such a way as to allow me to get back to a safe state if something goes horribly wrong). What I can do, however, is make a map of the trouble spots that I do discover.

I recommend really strongly doing these explorations very early in the testing cycle. Take the application and poke and prod it in ways that may be totally foreign or unique to you (possibly call in a colleague who doesn’t have as much experience with the application and let them tweak it in unconventional ways). The net result, hopefully, is that you will discover some land mines. Often, these land mines are the very ones that were set years earlier, and the developer that wrote the code may have long since moved on to other areas or another company (note: I'm not trying to disparage developers with the phrase "Land mine" or say that they are destructive or the enemy. Far from it, but the gotchas that are hidden in even some of the best designed code are, indeed, potential hazards that deserve to be explored before someone else innocently finds them). Doing this early in the cycle can give the development and management team ample time to consider whether or not they will go the extra steps necessary to do a full sweep of the area the tester has uncovered. Sometimes, the land mines are clustered together, and fixing an areas takes care of a bunch of hidden problems with limited repercussions in other places. However, there are times when setting off to fix one issue causes a chain reaction and other land mines then become much more clear.

Many times, there are no easy answers in these scenarios, but at the end of the day, we who test need to learn to not fear the repercussions of finding land mines. We need to fear a customer stumbling into the mine field more than we fear ourselves stumbling in. We can be armed and we can isolate these issues before they have the potential to become a dissatisfied customer… and a dissatisfied customer that vents on the Internet can be *way* worse than a software land mine. By doing our best to make sure that our development teams understand these risks, and that we spell out the map as clearly as we can, they can then proceed with the clearest information possible and make informed decisions. Should those informed decisions also include steering clear of the land mines, well, we just have to accept that… and perhaps remind them of where they are again later, when there may be a better focus to clear the no man’s land one again.