Selenium Conference, Day 3

Day 3 got underway with an early arrival to set up, check the sound board and make sure that we could get more recording done (I have approximately 30 pieces of audio to go through, so it might take awhile to parse through and massage all of it). David Burns was kind enough to sit down and give me an interview discussing his involvement as a committer and his views of both the present tech and future of Selenium and his involvement with it. also, for those interested, David Burns is the Author of the Selenium 1.0 Testing Tools Beginners Guide that is the subject of my Practicum section of my blog :) ).


Wednesday morning's keynote was given by Bret Pettichord, he of the legendary cowboy hat and the Watir evangelism, among other things :). Bret lead a tour-de-force performance championing science and eschewing the drugs that are record and playback. The dark ages of test tools actually had gag orders; you couldn't criticize the tool vendor if your system didn't work. Much discussion of Watir and Selenium and the key quote that I found very valuable, and it seems a lot of others did, too: "Watir was made to help make testers better developers. Selenium was made to help make developers better testers". Again, Bret followed on with the example from Jason Huggins yesterday, and did his whole talk without any visual aids. It seems a lot of people appreciated this approach as well. The key takeaway is that the story we are telling about Selenium does not match the reality of what we are experiencing. If we look at the Selenium site, it's all about the Selenium IDE and recording tests. If we look at the people here at the conference, almost nobody is using the IDE. Our story does not match the science. We need to change the story, and help testers realize they must make the step from tester to programmer. They don't have to be brilliant programmers, but they need to know some underlying concepts and not be afraid of writing test programs.


Andreas Tolfsen followed on with a talk about how Opera is using Automated testing in their organization, and shared that they generated anywhere between 1,000,000 and 6,000,000 test results *EVERY DAY*! Another interesting statistic? Their developer to tester ratio is 2 to 1! While I had a chance to catch a few of the comments from this talk, I was mostly engaged interviewing Ashley Wilson for the podcast. For the record, everyone at this conference owes Ashley a huge vote of thanks! She has very much been the "wizard behind the curtain" for this conference. Most of the moving parts and actions that have been going on to make this work have been primarily handled by her and a troop of volunteers, so if you see Ashley, definitely say "thank you" for what has been a terrific few days.


Eran Messeri followed on with what has vexed and interested me for some time... how can we test in virtual environments better? Google has a huge initiative regarding how to run well over 1 million test cases on a variety of platforms. Their answer is a heavy focus on using virtual machines that they can spin up and down, as well as massive parallelism of test execution, using what Google refers to as "Test Shards". Not only are tests run on multiple configurations, but test suites are literally split up to run over dozens or even hundreds of virtual machines. Eran insists this is not a holy grain by any means, there's a number of drawbacks to this approach, not the least of which is a continuous need to maintain many very large disk images and update the applications on a regular basis. Still, even with that, the benefits outweigh the disadvantages considerably.


Francois Reynaud and Kevin Menard discussed the changes and the opportunities for testers with Selenium 2.0 Grid. There are some exciting changes in play with Grid 2.0. The most interesting one off the bat is that it's just one jar file now, and therefore it is now easier to install and set up. Additionally, Grid 2.0 can also act as either a node or as a switch, it also has a lot of customization options. Oh, and the demo looked really, really cool :). What's more, Watir users will be able to leverage Grid 2.0 to extend their tests across multiple machines via watir-webdriver. The load balancer in Grid 2.0 allows the user to customize which requests are sent to which nodes. Also, Grid 2.0 is not bound to Selenium. Excited? Alas, we'll have to wait until Selenium 2.0 beta 4 is released, but beta 4 is practically all about Grid 2.0. Can't wait :)!!!


Andy Tinkham followed up with a talk about using Selenium and Cucumber to demonstrate a case study of how his Health Services company is structuring their test cases. They are doing as much as possible to separate the data from the tests, using random test data to increase the odds of finding bugs, and using Cucumber to help construct the tests in plain text while still making the underlying framework effective for those using it. The Cucumber tests and features are constructed with QA, Dev, BA's and Product managers to create the initial stories. They use just enough source code so that they can translate plain text into test calls. Nice example of a case surrounding patients with allergies and notifying users that this is the case. The scenario as described (given, when, then) is well constructed and sensible to anyone who reads it (as is the promise of Cucumber, generally speaking :) ). Seriously, this is an impressive real world example and while some of this is over my head, it's looking to be less over my head. Additionally, I like seeing the references and comments to meta-programming using Ruby (since the company I work with is heavily Ruby, I like hearing more and more about success stories of Ruby working well with Selenium and Cucumber. It gives me hope :).


Following lunch, Joel Klabo talked about "Flex Piloting". First thing to note, Joel is a 1st time speaker, and I want to offer my congratulations to him for stepping up to speak. as he said when he forwarded his talk for consideration "you have to start somewhere". Flex Pilot is an add-on that can be run in both selenium IDE and Selenium RC. it's an add-on to both, and allows for an additional series of commands top be added to both. Demo looked interesting and it has sparked my interest :).


Next up was the ADD portion of the program... that's right, Lightning Talks. Seriously, I'm not going to do this in real time, I just don't type that fast (LOL!), but suffice it to say that the presenters are both entertaining and covering a broad variety of topics (Selenium Tips in poem form? whoa!). Also, as I was watching the 2nd talk I noticed that Devon Smith was delivering it... I had no idea she was here! Devon, stop by and say "Hi", I'd love to finally meet you in person! Amazing and broad litany of talks presented and only a couple of buzzes. thank you all for giving me my "shiny fix/short attention span theater" for the day!

During the break, I finally cornered Jason Huggins and got an interview (seriously, I've been trying to get this interview for three day, so I'm a little psyched :) ). Following the break, Adam Christian gave a talk about Automation Battle Scars. The most memorable opening slide ever... "Automation Prison Rule 1: Punch a developer in the face!" No, not really, but don't be a pansy. Get in, make your case, make your stand and get to it. Earn the respect of the development team. Stop trying to take shortcuts. Learn how to code. You don't have to be an expert, but you'll be well positioned to do better testing and better automation work if you learn some of the basics. Please note, much of the tone of the talk is tongue in cheek. All in all a great and entertaining talk.

Simon Stewart is the final speaker, and it brings us back to why we are all hear... what was the easiest way to control a browser. It was a good goal once upon a time, but the ambition has changed. There's way more to the web than there was 7 years ago. So what's next? We need to desroy the project to make the world a better place. We need to make the effort to reinvent and redevelop Selenium for the future. We are going to be the ones to help do that, and browser vendors are stepping up to help make their part of that happen. We need good high quality bug reports, but much more important, we need good quality patches. the future lookks amazing, the watir project is going to be based on webdriver, so is selenium. The browser community is getting in on the game, and helping us get to the point where standards are set. We're getting speed and performance increases, and we are getting a heck of a combined community. there's a lot that we will be able to do in the future, but note the point, it's what WE will be able to do in the future. The beauty of open source projects is the WE. the community needs us, all of us. Let's all commit to doing something to make Selenium and watir and web Driver and [fillInTheBlank] even more incredible than it is today.

My thanks to everyone for a tremendous program and an amazing three days. It's been an amazing experience. Here's hoping I can bring this all home and put it into practice in a meaningful way, and be one of those WE's that the community needs. Now I'm hoping that we will have some fun this evening in the closing hours, and I'll talk about that in another post.


Afternote: Some have wondered why I haven't posted a review yet for Selenium Simplified, Alan Richardson's book. The answer is, I have, but it's in a different place. You'll find my review for Selenium Simplified in The Testing Planet, the journal published by the Software Testing Club. Paper copies just went out this week, so those who purchase the paper version are getting first crack at it, but I am willing to bet the electronic version will be available by the end of the week, and then you can see my review there.