BOOK CLUB: How We Test Software At Microsoft (2/16)

 This is the next installment in the "book club" approach to reviewing "How We Test Software at Microsoft". This installment covers chapter 2.

Chapter 2. Software Test Engineers at Microsoft

Microsoft didn’t have dedicated testers until several years into the life of the company and even then, as a distinct a career part it took time for that to develop. Microsoft claims that a high school intern named Lloyd Frink was the first tester, hired in June 1979. The first full-time tester is said to have been hired in 1983, with a greater emphasis on hiring testers starting in 1985.


Usability was one of the first testing disciplines to come to the fore, specifically around aspects of Microsoft Word such as Mail Merge and other features that were more challenging than others and required more detailed testing to make sure that the system behaved as expected and was accessible to the users.


Microsoft's developer title is officially Software Development Engineer (SDE). The formal title for software testers at Microsoft is Software Development Engineer in Test (SDET). Microsoft considers testers to be developers, and an expectation that testers know how to code is emphasized. Testers design tests, influence product design, conduct root cause analysis, participate in code reviews, and write automation. Microsoft sees its biggest differentiator from other companies being that they deliberately seek coders to be testers, not necessarily because they are looking for automation experts and to automate all tests, but because testers who understand computer architecture and software development will have stronger analysis skills than those testers that are not as adept at coding.


Microsoft does not specifically hire subject matter experts in many areas, thought they do a hire some. The testers are actually trained to be an SME for the product area they will be working with, in addition to learning how to adequately and effectively test the application. Microsoft likes to focus on what is considered the “Tester’s DNA”:

"Tester DNA has to include a natural ability to do systems level thinking, skills in problem decomposition, a passion for quality, and a love of finding out how something works and then how to break it," he put the marker down and looked at the room. "Now that is what makes up a tester that makes them different from a developer. The way we combine that DNA with engineering skills is by testing software. The name we choose should reflect this but also be attractive to the engineers we want to hire. Something that shows we use development skills to drive testing."

–Grant George


Now, I’m going to opine for a minute here (hey, it’s my review, I’m allowed :) )… in some ways, I think what Microsopft has done is brilliant, in that it allows a path for software testers to work towards and develop solid coding skills, and that is good if that’s what a tester wants to do. However, it has likewise been picked up by many other organizations and a “me, too” attitude that testers will be coders, with little understanding that not all testers, even technical and seasoned testers, necessarily want to be software developers, or for that matter need to be. Many of us don’t and we look at coding as a skill we need to be aware of and have some knowledge about, but not necessarily be the prime focus of our efforts. Nevertheless, that is the approach that Microsoft takes, and that’s reasonable for their organization and their goals.


The authors state that one of the primary motivations in this direction was the lengthy licensing and support agreements they had for software and applications (3 years at the consumer level, 10 years at the corporate/enterprise level). There’s also no question that the SDET title is a strong recruitment tool; many people who develop software and may choose not to go into full time software development may consider the SDET track to be a good one to follow.


Microsoft looks for what it considers to be its “ten core competencies” when it hires engineers, regardless of whether they will be SDE or SDET:


• Analytical Problem Solving Can the individual in question solve difficult problems?
• Customer-Focused Innovation Does the individual care about the customers and look to see how to help solve their challenges?
• Technical Excellence To testers understand the underlying operating system and networking code, and do they understand how to optimize code?.
• Project Management Can tester sdemonstrate appropriate time management and prove they can deliver on their goals in a timely manner?.
• Passion for Quality In short, you really have to care about product that works and works well.
• Strategic Insight Can you help find that next big thing that shapes the future?
• Confidence You may get pusback from developers, can you stand up to it and push back as well when it’s warranted?
• Impact and Influence Are you an agent for change? Can you articulate how you are?
• Cross-Boundary Collaboration this translates to “how well do you play with others”?
• Interpersonal Awareness Can you be critical about your progreess? Do you see areas that need improvement, and once you identify them, will you act to make the improvement happen?


The SDET role at Microsoft is considered to be one of the most important we have, and carries an enormous amount of responsibility in shipping the products we do. When I first start talking to the candidates, I really want them to understand that testing is just as much a science as writing code and understanding the algorithms. I start talking to them about the wide range of things that testers must be able to do.

--Patrick Patterson, Microsoft Office Test Manager


Microsoft says that about half of their hiring comes from college graduates, the other half from within the software industry. Their ideal industry candidate is someone who works where the role of product developer and software tester are combined. Microsoft’s developer to test ratio is astounding, approaching 1 to 1. This is a far cry from other industries, where the ratio of developers to testers is much higher, often 5 developers to 1 tester or even 10 to 1 or greater (in the spirit of full disclosure, the company that I work with has a ratio of 8 to one, but then I am the only tester at my company… I told you this comparison would at times be mind boggling to me :) ).


So what does Microsoft recommend for the aspiring software tester that wants to become a SDET? After employees go through what Microsoft calls New Employee Orientation (NEO), each tester works with the Engineering Excellence (EE) group to receive additional technical training. “Testing at Microsoft for SDETs” is usually taken within 12 months of starting at Microsoft. This is a 24-hour course, and much of the topics are covered somewhat in “How we Test Software At Microsoft”. From there, a significant amount of training is a vailable for the SDET to advance and develop along their career path, with much of the training supplied in-house by Microsoft, in the way of software development classes, languages, architecture, and advanced testing topics.


Microsoft has basically two main roles, individual contributors (IC) or managers. Depending on the career path chosen there are various “career stages”. Some testers try out management, and then decide to come back to being IC’s. Often, senior engineers jump back and forth between being IC’s and managers.


Microsoft adds somewhere in the neighborhood of 500 testers every year (AOP). The 9,000 test engineers at Microsoft play a large role in helping to ensure that the products ship with high quality. The variety of opportunities and the ability to grow within the company is limited only by the individuals own abilities to rise and meet the challenges.


We will pick up again with Chapter 3 on Saturday. See you then!