One of the challenges being a lone gun tester is the fact that, often, you don’t have someone else to ask questions with. Sure you can talk to developers about issues and areas you have concerns about, but that’s not what I mean. It’s rare that time will allow a person to consistently sit down with a developer and just ask broad and open-ended questions about a product, a technique or an idea. Larger test organizations allow testers to have this opportunity. Frequently, the Army of One tester ends up doing most of their thinking or brainstorming alone… but they don’t have to.
Today I had a cool experience. One of our domain knowledge experts had some time today and asked if we could set up a pair testing session, with the idea of “asking the product some questions”. The domain expert in this case is an Attorney very well versed in Immigration Law. There are a lot of layers to testing software that services the legal profession, which my company does. While I know a fair amount about Immigration and Employment Law just by virtue of repeatedly testing and looking at the challenges our products are meant to address, I will not have the same level of experience or expertise that a dedicated attorney would have.
As we sat and tested, we discussed a number of aspects about the product that looked “interesting” (that’s a euphemism for “something’s not right”). Many of these situations consisted of items and pages and applications I’d looked at countless times, and felt they worked as designed. They did, but the domain expert can often see things in ways that a tester can’t (at least at first, testers are notoriously fast learners :) ). While this is helpful, the most helpful aspect is the ability to sit down and ask questions of a product, and more specifically, ask questions a user might ask themselves, even if many of the questions seemed far- fetched or unlikely.
Through half a day of testing, I discovered areas I was sure I was covering well, but realized I could do lots better, and some areas that I could just flat out ignore (well, not completely, but certainly give them less emphasis for a time). Having this resource on a regular basis can be a blessing, but it's not practical at certain places. That’s OK, because a little can go a long way. It may be impractical to ask many of the people to participate daily, but given an option of once a month or so, you may be surprised at how many people are willing to sit down and roll up their sleeves and muck around for a half a day. Spread out among enough contributors (developers, support staff, marketing, sales, etc.) you can get a lot of unique input to help energize testing efforts or, at the minimum, consider questions you may not have thought to ask yet.
1 comment:
Great post, thanks for sharing this experience.
It also goes to show that what we think about as test coverage is an illusion. The only thing we can say is that under these conditions I tested x, y and z and at the time it worked once or a number of times.
I've been in the same situation where someone with more domain knowledge than me pointed out why some areas are important to the end users which I blatantly ignored or skimmed through. It's a humbling experience and a good eye opener/learning opportunity.
Post a Comment