Over the next few weeks, I hope I'll be able to answer that, and to do so I'm going back to the BOOK CLUB format I used last year for "How We Test Software at Microsoft". Note, I'm not going to do a full synopsis of each chapter in depth (hey, that's what the book is for ;) ), but I will give my thoughts as relates to each chapter and area. Each individual chapter will be given its own space and entry. This entry covers Appendix B.
Appendix B - 25 Tips to Reduce Testing Cost Today by Catherine Powell
OK, wait a second... didn't I review a batch of "25 tips" yesterday? What gives?! Well, Appendix B is a strikingly similar set of 25 tips. In fact, they have exactly the same titles. The difference? Many of Matt's tips had to do with things that cut waste, but were organizational, and possibly out of the realm of the Individual Contributor Tester (i.e. the Tester doesn't have the authority to do many of those things). In other words, if I were an individual Tester, what could I do to lower costs where I don't have to ask anyone's permission to make an impact? That's where this "list of 25" comes in :). Look at them from the perspective of how any Tester can put these into play and make them effective. Also, don't try to do all of these things at one, do one at a time, practice it, master it, then move on to something else. Think of these like a pebble going into a jar. It may not seem like very much, but with enough pebbles, you'll ultimately fill the jar ;).
Tip 1: Cut your documentation to a minimum
Let's do what we can to make our documentation more DRY ("don't repeat yourself"). Have you been copying and pasting things into your docs? Stop that. "Incorporate by reference". Replace those copies with a simple note: "See XX", and then put it - just once - at that location.
Tip 2: Make the cost of changing the documentation cheap
Source control, wikis, whiteboards, they all allow us to share documentation and change it quickly and from multiple sources. Leverage that. Make internal reporting as quick as possible, and let the QA Manager or lead handle the more formal report (which should also be easier to do with this collaborative apprach). Start with your own team, then once you've shown the benefit by collaborating on the documentation within your team, share it with other teams and encourage them to follow suit.
Tip 3: Never be blocked
Make a wish list of the things you wish you could do but you don't have time for on your whiteboard, a sticky note, or in your Smartphone. Spend just ten minutes on this, you don't want a massive list. Then put the list aside. Next time you're blocked, pull out your list. Now's the time to do something on it!
Tip 4: Eliminate Multi-Project-ing
Individual Testers may not be able to choose if they work on multiple projects or not, but you can choose how you deal with working on multiple projects. Pick a single project and create a meeting in your calendar for you and that project - and spend that block of time working on that project only. The more often you can do this, the better. Minimize task switching. Turn off email, IM, phone, put a “do not disturb” sign on your chair while you are testing. Whatever it takes :).
Tip 5: Automate entirely redundant processes
Automation freaks some people out, because it sounds so huge and nebulous. Try some "computer aided testing" instead. Make a shell script or a batch file. Find a repetitive task that you've done at least twice in the last month, and create some "computer aided testing". Then check into the source repository so everyone can use it. Start small, get some ideas together, and build it out as you do more of it. Even a list of commonly used commands in a single file can be a huge boost.
Tip 6: Start with Soap Opera Tests
Soap opera tests are where you throw everything, including the kitchen sink, into your tests. Load them suckers up and let 'em run. You might find some spetcular issues :).
Tip 7: Start with quick attacks
Quick attacks are finding ways to do something hairy and over the top to inputs or outputs. Search for eveything. Load Hamlet into a text field (thank you, QAHatesYou, for my ultimate favorite Quick Attack ever :) ). If you feel fancy and want to go a little white box, then find every error message that a program is supposed to display, and force the program to show it. That's one of my all time favorite tests; lots of time is given to the happy path. Have they spent nearly as much time and effort on error handling? This is a great way to find out :). You get the idea.
Tip 8: Test Early
If you're in an agile environment, go through your last iteration and identify how long it took you to start testing each story after it was done, and come up with one idea to start each one faster. If you're in a more traditional environment, go find your friendly local developer and offer to spend half an hour working with him to "pre-test" his feature together. If the developer looks at you funny, offer to bring donuts (or beer, some people are just motivated differently ;) ).
Tip 9: Develop a regression-test cadence, and keep it light
Creating a "testing-oriented system diagram" so you can figure out what components relate to what other components. Use it to identify what needs to be regression tested and what you can more safely not run. Ask your team to expand on the drawing, discussing what fixes or feature changes might break other things. Have an architect validate your diagram. Again, if necessary, bribe with donuts or beer ;) ).
Tip 10: Test the things that actually yield bugs
Pick one area that is on the top of your "must regression test" list, and one area that is on your "we don't have to regression test" list. Then do it, with just one area and with just one release. Prove to yourself that it's okay to not regression test *everything* every time. Focus your testing efforts on areas that are at higher risk of having new problems.
Tip 11: Elaborate - and communicate - a test strategy or triage strategy
Create a status update that you can publish frequently. Publish it at least twice a week for the rest of the release. If your release is in under a month, publish the status update daily. Be sure to include the "must fix bugs" and a summary of new bugs found.
Tip 12: Decrease your time spent reproducing the problem
Next time you find an unreproducible problem, set a timer (an hour or two max). Now attempt to reproduce the problem and gather information. When time is up, send the unreproducible bug over to development with two or three ideas for extra logging or information that will help track down the problem if it happens again.
Tip 13: Do a low-cost beta program or pre-release
First, do a beta test of one. Get an account rep to bring a friendly customer to your offices. Let them use the newest software for an hour or so. Sit next to them and watch what they does and what questions they ask. Have a frank conversation afterward (thoughts, likes, dislikes, what's missing, etc.). Next, do a ride-along for half a day or a full day w/ an account rep to a friendly customers site, and shadow the customer(s) for half a day or a day. Keep quiet and take notes about how the customers actually uses the software. Share this information with your team. Learn from your actual end users. If the customer is open to it, record them with your phone, or at least take pictures. Pictures are worth a thousand words.
Tip 14: Recognize and eliminate project wankery
This might be hard, but give it a try anyway. Think of the meeting that you feel is the biggest waste of your time. Then make plans to skip it. Provide whatever informstion you need to satisfy those who feel you miust be there, but explain you can't make it for a pressing reason (no need to elaborate). The goal is to prove the necessary work *can* get done and that the product won't suffer if you don't sit in this meeting Swapping a meeting for a brief written report is a good bargain in service to your overall goal. You may not be able to do this very often or for many other meetings, but even if you can get out of just one of them on a regular basis, over time that can add up to a lot of quality testing time.
Tip 15: Stop fighting to manage the project
#1 Rule: NO WHINING. If you don't want to manage a project, don't complain about how it's being managed. It's probably not part of your job. Don't let the things other people choose to do interfere with the things that only you can do. Let it go.
Tip 16: If you're going to fight, win
Sometimes the decision really does matter! If it's truly important, you need to be able to sway the decision your way. Your reputation, your argument, and how much support you’ve garnered to your side will have a huge impact on the outcome. The next time you make a decision, sit down and write out why you're making that decision, along with the pros and cons of the choice you are making. Then give that explanation to your boss and ask for feedback. Make clear this is *your* decision, you're not asking your boss for permission. This lets you see if you argument is persuasive and if others will back you up on your decisions, and if you are really ready to "fight to win".
Tip 17: Walk around and listen
If you're a test lead or a test manager, take fifteen minutes and walk around the office in the morning. Repeat your 15 minute walking circuit that afternoon. Don't interrupt anyone, and don't talk with them. (It's okay to say "hi" back if someone greets you.) Just watch and listen, and notice who doesn't seem to be doing anything either time you walk by. After your circuit, sit down for 10 minutes with everyone who was not accomplishing anything and ask what you can do to unblock them or help them find a project to do.
If you're not a test lead or manager, see what you are doing with your time... no really, what you are really doing with your time? You may find that you are not nearly as focused as you think you are. Be brutally honest with yourself, and ask, OK, what can I eliminate? What can I minimize, and how can I get the necessary project time beefed up and finish what needs to get finished? Aim to decrease the distractions each day and boost the project stuff each day until you get to a realistic level of external tasks that have to be done, and as much project focus as you can reasonably perform. Seriously, nobody's going to be able to be 100% during an 8 hour day, but try to maximize where you can.
Tip 18: Write test automation that attacks the business-logic level
If you are using GUI tools and automating through the GUI, detemine how you might use another tool or rewrite your test to do a similar job from under the board. This might take some coordination with the development team but it certanly can be done. Determine what internal tools they use to get to the console or other hooks in the program, and then start working out tests that utilize them.
Tip 19: Develop a test automation library
This may be tougher for non coders in the test group, but think of the scripts and the automation you may already have. How much of it is duplicated in other tests? If you see a bunch of duplication, here's a chance to make a library or somehow centralize those items so that the can be called in a simpler manner. Personal experience: a neat trick in Cucumber is to take commands that get run a bunch of times and make "macros' of them by using the %Q|[cucumber line]| syntax. This lets you group various Cucumber commands into a one line call. DRY in action :).
Tip 20: Develop or hire expertise
Spend 15 minutes and figure out the expertise of each member of your team. Then take another 15 minutes to figure out where you need experts that you don't have. These are the areas you should be considering for training or hiring: to fill your expert gaps.
Tip 21: Get a return from conferences or don't go
Play "find an expert" the next time you go to a conference. Find someone with experience in an area your team lacks. Follow up with the expert - a quick email is fine - to establish a relationship. When you have a question about [fill in the blank expertise], you will be able to reach out to the expert. They don't have to be the famous testers. Often the ones that aren't are totally down with helping you and will often bend over backwards to help you get where you want to be. Note: don't abuse this relationship. They won't do your work for you, but if you do your homework and come to them with specific questions and showing you've already given your all, they may prove to be surprisingly helpful :).
Tip 22: Build a social network
Pick your favorite testing site (Software Testing Club, StackExchange, whatever) and answer one question. Next time you have a question, go to the same site and ask it. Do this regularly (maybe once a week), Over time, you will build a network of testers you can rely on. You'll also build a reputation and have public evidence of your own competence, which can be huge when it comes time to look for a new job :).
Tip 23: Examine invalid bugs carefully
Do some data mining in the defect tracker. Create two reports: (1) number of bugs marked invalid by reporter; and (2) number of bugs marked invalid by feature/module. Take that report and go talk to the reporter with the most invalid bugs. Pick a handful to walk through with him and spend 30 minutes brainstorming how to decrease that rate. Then take the report to the owner of the feature/module with the most invalid bugs. Pick a handful to walk through with him and spend 30 minutes brainstorming how to increase team-wide knowledge of his module/feature so the number of invalid bugs decreases [I'm not going to paraphrase this one, I think it's perfect as is :) ].
Tip 24: Build a test model that deal with the natural up and down staffing need for test resources
If you're already a tester on a team, spend 15 minutes brainstorming "what do you do when you're idle". A list of things testers can do when testing on the product is relatively light should be the result. Make a list of concrete actions you can perform. Next time you're idle, grab something from the list, and do it.
Tip 25: Ask the team to identify opportunities to drive out waste ... and implement them
Is there something in your processes that seems totally stupid and a complete waste of time? Kill it! Are there documentation steps that no one ever reads or does? Remove them. Tests that add little or no value? Weed them out. This is a gift that keeps on giving. As soon as you get good at this, you will find other areas you can trim away, too, and the better you get at it, the more areas you will discover. This is also a cycling list. As you cut wasteful activities, you will find room for more fruitful ones, and that fine, but remember, no fruitful idea will remain 100% fruitful forever. New processes develop waste of their own accord, and soon you'll be fining ways to "refactor" there, too.
No comments:
Post a Comment