WTA19: Bulls and Cows and Exploratory Testing

First, I want to take a little time to say thanks to Albert Gareev, who has for all practical purposes been my right hand dude when it comes to all things Weekend Testing Americas. this week was his first chance to fly solo, necessitated by the fact that I really couldn't get into a comfortable position to type for an extended period (we're working on that situation this weekend, btw). Albert ran the session himself, kept everything on track, and the session itself was fun, too.

This is not an Experience Report per se. well, it's my experience report being on the outside looking in for a WTA session. I enjoyed being "the tester" again, and being able to actively participate in the challenge. There's a site called http://www.pencilandpapergames.com/ and one of the games is called bulls and cows. It's a simple number guessing game. The system will pick a number and you will pick a number. You define the number of digits and then you pick a number. Based on the number you enter, the game will respond back with "x bulls, y cows", where bulls represents a correct number in the correct position, where cows represents a correct number but in the wrong position.

Our goal was to approach it as an exploratory testing exercise and see if, through our notes, we could explain the methods we used to solve the puzzle. Many of us used different techniques, and I'll let others blog about their approach, but I'll share mine. My goal was to see if I could determine the correct numbers and their placements as quickly and with as few guesses as possible. To that end, I worked out the following strategy:

1. enter various 4 digit values in quick succession with minimal repeat of values (2435, 6179, 5862, 8765, etc.) It may have been luck but I was able to hit a cow on the first turn, a cow on the 3rd turn and a bull on the 4th.

2. Looking back, I saw that the cows and the bulls all had the number 5, so i went and played around with numbers around 5, including "5555" which gave me 2 bulls, 0 cows (ah digits can repeat!). Further tinkering showed me that the 2nd digit and the 4th digit were 5s. from there I tried to see if I could get other numbers to indicate if I could nail down cows (number values) for the first and third digit, thus I cycled through [1515, 2525, 3535, 4545, 6565, 7575, 8585, and 9595]. All gave me the same answer, 2 bulls, 0 cows.

3. Well, that's weird, I went through all the possible digits... oh no, wait, I didn't, I forgot about "0". OK, just for grins... "0505"... and with that, I got 4 bulls!

This is a classic example of using exploratory testing, based on my own context and based on my own situation at this given time. Another set of numbers could be picked, and were I to run the exact same set of steps, I would not have gotten the same results. In our everyday testing, we also will find that making a script for our tests will help us one time, and may make for a good rule of thumb as a basic checklist, but if I played the game again, I'd need to approach the problem slightly differently.

Some of the examples were much more complex, and went into far greater detail than mine did, but the basic premise is the same. the notes we make, even when we play a simple game, can tell us a lot about how we test and how we can improve the process.

Again, my thanks to Albert for leading today's session and letting me for all practical purposes a lurker today. It's good to know that Albert can fully run these sessions and I don't have to worry. Albert and I would likewise enjoy some company in having more people ready, willing and able to facilitate sessions. If you're interested, by all means let me know.