SummerQAmp: What is Bias? Intro

This is the next module that I am working on for the SummerQAmp program, and as such, I'm looking for both feedback and a way to see if my ideas make sense. Read into this what you will :). 

Bias is a tricky topic. There are several examples of it, and ways that we manifest it. Each of them is somewhat challenging to describe. One of the ways that I want to direct this is to use a thought experiment using a game many of us are familiar with already. 

I did this as a session for Weekend Testing some time year ago. In it, we used the game SET. Specifically, we used two versions of the game. The first version, which can be seen here, has a basic format and set of rules to play the game. In it, a single player tries to find six matching sets that follow the rules of the game (the matching criteria is that everything has to match or nothing can much for it to be a set; same/different color, same/different number, same/different shape, same/different shade). 

We used this approach to help get the players familiar with the game dynamics and the basic rules of how SET works. From there, we let the players loose on a different version of the game, a multi-player version that followed the rules for matching, but the way to get scores and find sets was different, in that people were competing to see who would find matches first.

What was interesting to see was that a number of the participants were reporting issues they were having with the second game. The issues were legitimate discussion points, and had a lot of validity, but what was more interesting to me was that many of the issues being reported were because they were comparing the second game's dynamics to the first game. Why? They were two separate games, and they had two different sets of rules. One game didn't necessarily reflect on the other… except that it did, at least in the mind of the participants in this Weekend Testing event.

The reason? Playing the first game trained them to consider interactions one way, and then playing the second game didn't match their expectations. For some, that was a frustration point. They wanted the two games to play the same, or similar enough that the dynamics of the first game would be reflected in the second.  In short, I unintentionally "biased" some of the participants by having them play the first game and get used to the "rules" before I had them play the second. 

What if I had said up front "these two games are very different, and you should not compare the two to determine acceptability of play"? Would the outcome have been different? It's hard to say, but the point is, I didn't. By not doing so, I introduced bias into the process.

There are many areas of bias in software testing, and may ways to approach it. This is one I'm planning on using. What do you think? A reasonable way to introduce the topic? Could you think of a simpler way? I'm serious, I am totally open to suggestions. There will be more to come :).

Michael Larsen, TESTHEAD