Exploring Black Box Machines

On Saturday, I had the chance to step back and be a participant again for Weekend Testing. Albert developed and ran the session this time, and frankly, I think it was one of the best ones we've had.

This session was called "Black Box Machine", and it was based around, you guessed it, testing a truly black box device. We really didn't have any idea what we had or what it did. It was a screen with a gauge, a red LED, a green LED, 4 blue buttons and 4 yellow buttons. Click on the experience report link above to see the machine layout we used.

The buttons were functional, and combinations of the buttons controlled the lights and the gauge.

I had a chance to work with and discuss the approach with Russ Poole, a newcomer to Weekend Testing. Often, we split into pairs for these sessions to do the actual testing, with an experienced Weekend tester working with someone who is new to the format, or a more experienced tester working with a less experienced tester. We took some time to acquaint ourselves with the machine and worked through the buttons to see what they did. Through exploration, trial and error and questioning, we made a mental model of what the device might be. We knew the mental model might be wrong, but it was useful for a time to help us describe what we were seeing, and use a common language between us. With this approach, we were able to divide and conquer the problem and determine what the machine did. Well, determine what we thought the machine did.

Each of the testers or pairs of testers did the same thing in the time frame we provided, and each group or individual presented their approach. It was interesting to see that each pair had a slightly different method to their testing and their vision of the problem. Each of us used heuristics to help us define the problem, and to come to some conclusions. I explained to Russ that our mental model may be correct, or it may not be. Still, as long as we kept learning about the product and could try out our ideas, we were able to determine at least on the surface what we thought the machine was doing.

This exercise is built on a number of examples that James Lyndsay developed, the entire purpose of these "machines" being to help people understand exploratory testing in a way that may get lost when working with known products. With everyday well known software applications, we can use some markers and guidelines to help us, and those familiar items and ideas will naturally guide how we test. Tthese machines are void of any natural context, and as such, we are basically left to our own deductive and inductive reasoning to figure out what's going on.

Again, I think this was a fantastic session. My thanks to Albert Gareev for facilitating it, and my thanks to all of the participants for taking the time to come out and play with us on Saturday.