Where I Am Today: A Meandering Walk Through Twenty Years of Software Testing

As I made my way into the small loft on the fourth floor of an old brick building on Jessie Street in the South of Market area of San Francisco, I had a new opportunity to start over again with a different kind of company. When SideReel hired me, they were focusing on developing a solid Agile team and releasing software regularly. Like many organizations, they were heavily involved with Test Driven Development and doing much of their testing within their group. It worked in some areas but they wanted to have a better idea from outside of their group as far as testing was concerned, and that's where I came in.

SideReel, for those who are not familiar, is a television aggregator. It allows users to find, watch and keep track of television shows, movies, and WebTV programs. We have also made pushes to share that information with other users and help make it a "social" experience, so that, if you want to, you can recommend shows to others and let them know what you are watching. It's a pure web and mobile play, which means that the site is 100% online, and all of the assets of the site and software are deployed online as well. No software needs to be downloaded and installed, but browser compatibility and appearance is a big deal, and something I spend a fair amount of time with.

At the time, SideReel had been around for about three years, and had started in the living room of one of the founders, and had grown over time. While they were larger and had a bigger business than Tracker did when I got there, that same "family vibe" was definitely there, and I liked that a lot. I knew that I had a chance to be involved in something very different than anything I'd worked with in the past, and that was what was most appealing about the opportunity.

I had worked in many different types of environments, but Agile was a new experience. While Tracker had tried examining and implementing some elements of it into our process, this was the first time I'd seen a team that actively practiced it in most of their development work. I found the first few weeks when I was there, when I started working on test plans the way I always had, to be enlightening. Rather than having to spend so much of my time working with writing test plans that were scripted out, I was encouraged instead to examine the stories in the tracking system we were using, and to append my tests there. What? No over arching test plan? No long term back and forth over the quantity and details of plans? How crazy was that? It turns out, not so crazy at all. While it seemed odd to have everything associated with just an individual story, it made sense since the stories in question were sliced so thin that the functionality was encapsulated in those areas, and therefore my tests were realistically touching just a few areas; I could focus them into just the parameters of an individual story. I liked this model because it actually fit better with the notion I had of what  tester actually did, which was to provide information about what a product does, and allow me to focus on the areas that actually mattered. With the use of Continuous Integration for each check in, we had much of the regression areas covered (not all, as I would find out, but a lot of them). 

SideReel places a lot of value on exploratory testing, and it felt good to actually be able to apply that in a real sense. It's not that other places didn't value it, but that it was an area that wasn't really understood or given much focus in the past. Having a team that had taken the Test Driven Development approach to heart and emphasizing writing tests before writing code certainly helped in that regard. They also provided a large library of tests that had been written so that I could see what they were looking for at the code level and expand on it to see if there were other questions I could ask and get involved with.

This wild world of start-up madness, I would discover, would be short lived. I was only with the company for six weeks when I found out that SideReel was to be acquired by Rovi Corporation, which was itself an owner of numerous companies related to video an audio content, information and other properties related to entertainment (their holdings included Sonic Solutions, Roxio, AllMusic, Divx and several other brands). We were told that, while we would be part of a larger organization's structure and business model, our day to day operations as SideReel would largely go unchanged. While we would relocate to a new building and have our own area of that building, we would still be operating as though we were the same 20 people that were in the walk up flat on Jessie Street.

As we started to make the changeover, I was challenged by the development team to actively learn and get involved with writing automated tests for SideReel. My goal was to work from the idea that I didn't have access to the fundamentals of the code, and to focus on the front facing user experience wherever possible. The goal was to put together a battery of smoke tests that would give us confidence that any new features rolled in would not cause undue problems or challenges to our users. With that, I began to spend a lot more time learning about Cucumber, the underlying tools that worked with it (Capybara and RSpec), and learning as much Ruby as I could. This was a first. While in the past my ability to use automation and computer assisted testing was left to me to do as much or as little as I wanted, the goal of SideReel was to have me be as knowledgeable as possible of the test driven development approach that they used.

During this year, there was also a big emphasis on my part to write a lot more. TESTHEAD was one of those mediums, but so was getting published in various online journals related to software testing. It seems each time I had a chance to write something for one journal, another would contact me and ask if I could write for them as well. 2011 also saw the "How to Reduce the Cost of Software Testing" book be released and, needless to say, I was excited to be a part of that title and to be able to share with my management team that I was a contributor to that project. My goal was to show them that they had a dedicated and involved tester who was excited about being actively engaged in the testing community. I also enjoyed the opportunities that I had when various testers would come to the Bay area and we'd be able to get together and talk about developments and opportunities. I got involved with taking and then teaching the Bug Advocacy class, which is the second of the Black Box Software Testing classes offered by AST. I participated as a volunteer for the first Selenium Conference which was held in San Francisco, and developed two talks this year, one for CAST 2011 in Seattle and one for PNSQC 2011 to be in Portland. I also made a point of reviewing as many books as I could during the course of the year, with a number of publishers giving me access to lots of great titles to explore and learn from.

In a way, I had to deal with a slightly different challenge than I had dealt with at Tracker. Whereas I was dealing with overcoming boredom at doing the same thing over and over, the challenge I was now facing was keeping a balance between all of the amazing opportunities I was getting the chance to get involved with. When I told my team that I had been elected to the Board of Directors of AST, they were congratulatory, but I also spent some time talking with one of our founders about some of his concerns. While he was excited for me and happy to have someone so engaged in testing, he also had some legitimate fears that I was so involved and engaged that it would have a negative effect on what I was doing for them. While my goal was not to be distracted from doing my job, I could see it was entirely possible that too much testing could get in the way of doing testing (I know that looks odd, but it's actually true. He meant that I'd be so focused on talks, articles and AST business that I'd pay less attention to the testing needs that SideReel has).  I assured him that my goal was to do great things in the world of Software Testing if I could, and that that started with doing great things for SideReel. That was my first priority, and that it would be going forward. That actually takes some doing at times, but it also allowed us to set up some plans as to what areas of testing and blogging are "on the clock" and which areas are best spent after hours to explore and consider.

I've mentioned many times on this site the calamity that befell me in August of 2011, with a severely broken leg and the resulting recovery time. That experience actually took me out of the running for being able to present my proposed talk at PNSQC in October (it was still hard to move around at that time; I'd literally spent the month of September in bed or with my leg elevated). It did give me a chance to read more and dive into some areas of testing I hadn't had a chance to explore previously, and I had a chance to sign on for the roll-out of the third class in the Black Box Software Testing series, which was Test Design. This would be an interesting place for me to be, in that I was actually helping to teach a class in its pilot run that I had not actually taken. I had to review the materials a few weeks before the start of the class and get an understanding of the material so that I could help teach it as well.  It was a fun experience, but occasionally it felt like I was getting lost in the weeds. This was a big class and it had a lot of information to absorb, but overall I think we did a good job.

I'd be remiss if I were to let 2011 go and not talk about one of the initiatives I most enjoyed being involved in, and that was Weekend Testing. We put on two dozen testing clinics during the course of the past year, and I had a chance to actually discuss the process and do it live at CAST 2011. The real credit, of course, goes to the many participants from all over the world (not just from the Americas) that participated and got involved with our sessions, asked great questions, and offered suggestions for follow-on sessions.

2012 is now just underway, and my involvement with testing is stronger than ever. I look forward to a year where I will be able to learn more, get involved where I can and discover more opportunities, and in some small way be actively engaged with both my own company and with the broader community of testers in the Bay Area and in the world. The cool thing about software testing is that the only limits there are are the ones you place on yourself. The broader community is more than happy to help fellow testers learn more, do more, get more involved and be more engaged.

So what has this last year taught me?

- There's a huge benefit to standing up every day and saying what you have done and what your plans are for that given day. There's a level of accountability that document controls, sign offs an many other methods don't have. When you verbalize your commitments, they feel much more real and tangible, and you feel much more beholden to making sure they get finished.

- Agile methodologies do help to make testing more focused and help make the process of testing likewise more focused. Rather than having to deal with the changes to a product and having to test everything every single time, the ability to take a thin slice of the product allows the tester the ability to get closer to a more complete level of testing (note, I said more complete; it's still impossible to get to a totally complete level of testing, even in the agile space, but you can get  lot close to that goal when you don't have to deal with the entire product).

- Being an embedded tester within and Agile team has its benefits. It's nice to just be able to get up and show something to a developer sitting right behind me or across from me rather than having to invade someone's cube. It's a much faster process and a lot more effective.


- Automation is like any other skill. It takes time and patience to learn it, practice with it, suck at it at first, and slowly over time get better at it. I realize now that my biggest problem in the past was that I didn't really give myself enough time to absorb what I needed to do, not did I give myself enough daily practice to be able to actually have enough to absorb. Just like a batter doesn't go in and immediately hit home runs off a top notch pitcher, so a tester will not be able to write flawless test code the first time out. Start small, make some things work correctly, then branch out. Also, if you need a developer to help you, don't be afraid to ask, but go in with very specific questions, such as "which class should I be using if I want to test the ability of buttons that change wording while you hover over them? I've tried [A, B, and C], but so far, no luck." This shows you have at least done your homework and tried several things, and that you know the area you want to have them focus on and that it's a specific question. Under those circumstances, most developers are more than happy to help you get unstuck. If you are not prepared, or you seem unfocused in your questions, that's a lot harder for them to understand what you mean, and repeated requests will get a less than enthusiastic response back.


- Singers often fell prey to being under the influence of "L.S.D." back in my performing days. LSD in this case means "Lead Singer Disease". It's where they are so excited to be performing and getting involved in the process that they sometimes forget they are part of a band. When we get involved in the broader community of testing, whether that be speaking, writing, seminars, etc., we also can get very excited about the process, and forget that there's a company that's paying for us to do our day job.  When it's something like music and testing, it's easy to separate the two. When we are doing testing in our day job and writing about testing or doing seminars on testing in our off time, the lines do definitely blur. In this case, it's very important to know where one sphere ends and another begins, and while keeping them totally separate may be counter productive, you don't want to have to say that your testing is suffering because you are stressing over your talk you'll be giving at a local testing meetup. If it helps, think of being a musician in an original band and in a cover band. Yes, you're still playing music, but you have two gigs with two different requirements. Remember to sing the covers with the cover band and the originals with the originals band. Otherwise, it can get awkward ;).


So that's it... for now at least. Our riverboat ride is at its end. Of course, I hope to have many more years of software testing to examine and consider, and it's possible I may revisit this series each year to reflect on what I've learned and what you might find interesting as well. Until then, be well, and thank you for joining me on this journey.