A Book Commit: MetaAutomation #30DaysOfAutomationInTesting Day Two

I am writing this series based on the requirements for the "30 Days of Automation in Testing" series as offered by the Ministry of Testing. yes, I realize I am almost 18 months late to this party but work requirements and a literal change of development environments have made this the perfect time to take on this challenge.

Let's take a look at the Day Two requirement:

Begin reading an automation related book and share something youʼve learnt by day 30.

In a sense, this is actually a re-read but a first time full apply. Several years ago, Matt Griscom approached me when I was visiting Seattle and told me about an interesting idea he had for a book. We exchanged several emails, talked about a few things here and there, I looked over some early chapters and from that (and lots of talking to other lots more qualified people than me, believe me ;) ), Matt came out with the book "MetaAutomation".

Today, that book is now in its 3rd Edition and last year Matt gave me a copy and encouraged me to give it a read. Now that we are making this product transition over to C# and .NET Core (so as to work better with other teams who are already using that stack) it seemed a very good time to take Matt up on that offer :).

Here's a bit from the Amazon description of what MetaAutomation is all about:

"MetaAutomation describes how to do quality automation to ship software faster and at higher quality, with unprecedented detail from the system under test for better communications about quality and happier teams.

This book defines the quality automation problem space to describe every automated process from driving the software product for quality measurements, to delivering that information to the people and processes of the business. The team needs this to think beyond what the QA team or role can do alone, to what it can do for the broader team. Quality automation is part of the answer to all that is broken with “test automation.”

MetaAutomation is a pattern language that describes how to implement the quality automation problem space with an emphasis on delivering higher-quality, more trustworthy software faster. Much it depends on a radical, yet inevitable change: storing and reporting all the information from driving and measuring the software product, in a structured format that is both human-readable and highly suitable to automation. This change was not possible before the technology made available by this book.

Read this book to discover how to stop pouring business value on the floor with conventional automation practices, and start shipping software faster and at higher quality, with better communication and happier teams."

I have to admit, on the surface, those seem to be bold claims and from what I read in the previous versions, there's a lot to digest here. thus, I'm actually going to be trying something out during this month. If possible, I'm going to try to approach the rest of this month's activities, where relevant, by referencing this book and trying to see if I can use the principles in my practice. Thus, in addition to doing a full form book review at the end of the month, I hope to have had a chance to actually and completely internalize what I read here. In short, Matt, I'm not just going to read this book, I'm not just going to review it, I'm going to try my best to live it. Let's see what happens if I do exactly that :).

Aedificamus: Pushing Steel and Tereré: #HealthyTech30 Day Three

Day three. Let's do this :)!!!

This is another post in my Aedificamus health and fitness series and specifically for Saron Yitbarek's #HealthyTech30 challenge. Today I will be showing what may be a blast from the past and another way to consume yerba mate. Yesterday it was the hot and traditional approach. Today will be a colder but still somewhat traditional approach :).

Exercise: Let's Push Some Steel

I'm curious if anyone out there knows what these are ;).

If you are my age or thereabouts, my guess is that you have come across these a time or two in your lives. If you are sitting there thinking "you have got to be kidding, you actually have a BULLWORKER?!" my answer is "yes, and not just one, I have two of them!"

I owned one of these when I was a teenager and I used it for conditioning when I was on my high school's track team. Years later, when I was thinking about having an easy to use at home exercise apparatus I thought, I wonder if they are still around or if I could buy a used one somewhere. Turns out they are indeed still around and thus I purchased two of them, a classic bullworker (the bigger one) as well as a smaller model called a Steel Bow.

OK, for those wondering, a Bullworker is basically a compression spring between two sleeves, with a pair of handles, one on each end and a pair of cables connected to the handles. The goal is to compress the spring either by pushing the handles or pulling on the cables. I also have a set of extension straps that allow me to do broader range movements and attach it to a door. The smaller Steel Bow model also comes apart easily and allows the user to put in one of three spring tensions (light, medium, heavy).

Today I focused on abdominals, lower back, and shoulders. Whee!!!

These devices work on the concept of isometrics, where you contract your muscles and hold them in that flexed position for an extended period, as well as a progressive resistance push. Some movements you will be able to cover a lot of range, others you will feel like you are barely moving the device at all. If you are putting pressure and able to hold it, you are getting the benefits. Point being, if you go to Goodwill you may be able to find one of these. For compact, at-home exercise equipment, this is actually pretty cool.

Food: Yerba Mate: Tereré

Thicker cut tereré leaf with a typical mate gourd
(made from calabash) and a bomba for drinking. 

Yesterday I told you all about how I've grown to love chimarrão, a variation of yerba mate that is popular in southern Brazil. It's a preparation that is, IMO, best enjoyed hot. However, I hear some of you say, what if I prefer it cold? Does it work for that too? It can be but if you like your mate cold, there's another preparation that is popular in Brazil and several other South American countries, particularly Argentina, and that's what I'll be telling you about today, tereré.

First, tereré has a different flavor profile altogether compared to chimarrão. The leaves are dried using hot air and smoke from a fire, so they have a distinctive smoky flavor to them. Additionally, the leaves are thicker, so you can prepare tereré right in a calabash gourd or you can also prepare it and drink it straight from a canning jar (granted, the second way is a bit less elegant but still works.

Traditional prep method first. This is a mate gourd and a bomba that are both well suited for drinking tereré. Preparation is simple, take about half a cup of tereré leaves and put them into the gourd. take cold water or ice water and pour it into the gourd, up to the brim. Let steep for about 10 to 15 minutes. From there, place the bomba into the gourd and drink. The bomba filters out the thicker leaves.

If you'd like to do this without having a fancy gourd and bomba, here's my other method. Again, Brazilian and South American tereré fans, my apologies if this is borderline blasphemy but hey, it works :).

1. Take a canning jar and fill it halfway with ice.
2. Put 1/2 a cup of tereré leaf into the jar.
3. Fill the jar with cold water.
4. Close the jar and shake.
5. Let sit for 10 to 15 minutes.
6. Use a tea strainer and pour off the liquid into a cup.
7. Add more cold water to the jar and keep refilling.

After multiple refills, you will reach a dilution point but it will take a while.

It is also popular to punch up tereré by adding citrus juices (lime, lemon, orange, grapefruit) or pineapple juice but it is also very nice all by itself.

How healthy these are is open to interpretation but if you are someone who actively consumes soda or other beverages throughout the day, consider switching to yerba mate and see what you think. Hot or cold, I doth fully dig it :).

Late to the Party, Still Going to Jam: Day One, 30 Days of Automation in Testing

OK, here we go, better late than never, right :)?

Regardless, I have decided I want to tackle this one next because I am knee-deep in learning things related to code and automation and retooling with a new stack and platform.

I'm in the process of learning how to navigate my way around Windows 10 on a Lenovo Think Pad, getting my bearings with Visual Studio, and playing with some of the finer points of C# and .NET Core. In short, it's a perfect time to take this on and perhaps augment a tectonic shift in a work environment with some additional skills to help balance it all out.

This is Day One of a Thirty Days Challenge, this time focusing on Automation in Testing.

 Look up some definitions for ʻAutomationʼ, compare them against definitions for ʻTest Automationʼ.

Let's see what Lexico has to say about this (or at least what they had to say about it October 24, 2019):

automation: NOUN: mass noun

The use or introduction of automatic equipment in a manufacturing or other process or facility.

Origin: 1940s (originally US): irregular formation from automatic + -ation.


That makes sense, in the sense that that is a natural sense of the word. Something done to make something automatic. Granted, it uses the word automatic, which comes from the Greek word automatos:  acting of itself. Thus when we think of automatic, and by extension, automation, we are thinking of devices and processes acting on their own behalf. thanks, Lexico :).

So let's take a look at Test Automation. What does Lexico have to say?


No exact matches found for "Test Automation"

Interesting, and not really surprising, as it's not one word. Point is, there is no real dictionary definition for it. The closest thing we get is a Wikipedia entry (and yes, I'm a strong advocate of "caveat lector" when it comes to Wikipedia or any place else but there's nothing wrong with starting there and then extending the search. Wikipedia starts with:

"In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes." 

Where did that come from? 

Kolawa, Adam; Huizinga, Dorota (2007). Automated Defect Prevention: Best Practices in Software Management. Wiley-IEEE Computer Society Press. p. 74. ISBN 978-0-470-04212-0.

Hmmmm... not trying to be critical here but a book selling the best practices of software management gets to define what test automation means? Am I the only one who finds that somewhat amusing?

OK, to be fair, test automation is still fairly new (though it is as old as any procedural programming steps, really, so it's been with us since the 1940s at least). Ultimately it comes down to the idea that we have machines and we want to control those machines' execution and determine something about those machines' control that goes beyond them just doing that thing they do. Sorry if that's a bit breezy, I never promised you peer-reviewed scholarship ;). 

Seriously though, minus the snark, we are looking at doing a little more than repeat steps over and over. We're trying to determine a way that the steps being performed are actually being done right. thus, test automation goes a little bit beyond actual automation of steps to make a repetitive and repeatable process something a machine or pattern apparatus can do.

Aedificamus: Yoga and Chimarrão: #HealthyTech30 Day Two

I think I'm going to enjoy this feature :).

At this point, in case the hashtag picks up some new viewers, and if current viewers are wondering about the hashtag, I should probably explain for people who might be curious:

- "Aedificamus" is Latin and it (somewhat) means "building through practice". Latin aficionados will probably take exception to that but it's how I'm using it, so deal ;)!!! It is a tag and post title marker that I use here on TESTHEAD any time I talk about health or fitness.

#HealthyTech30 is an initiative that Saron Yitbarek mentioned on Twitter with the net goal that tech people like me get up and do something healthy for 15 minutes, at least, and then eat or drink something healthy and share it. If that's not screaming for blog post series, I don't know what is. Even if it isn't, I thank you all for indulging me ;).

OK, let's get to the exercise portion first.

Exercise: Fightmaster Yoga on YouTube

First off, I think anyone can benefit from doing a regular Yoga practice. Second, I particularly like Lesley Fightmaster because, though she is a good instructor, she doesn't present herself as polished or perfect. She has a laugh when her cat interrupts or people stop to watch what she is doing and that is just so human, endearing, and wonderful.

For those who want to try out her approach, she has a 30 Day Yoga for Beginners course that is terrific. I did the whole thing and yes, I would recommend. I'm currently going through her Playlist of 20 Minute workouts (because there is a lot of them and I find that length to be about perfect for me).

This is the one I did today.



Note: I have this playlist set to shuffle just to make it interesting, so I never quite know what I will end up with. You are welcome to play along with me or you can start at the beginning and work through in sequential order.

Food: Yerba Mate: Chimarrão

I hadn't consumed much Yerba Mate before my daughter moved to Brazil to serve her mission. A number of her companions are from the southern part of the country (Porto Alegre area), and they turned her on to "chimarrão" which depending on where you are is a blanket term for mate or this particular style of preparation.  She, in turn, turned me on to it and it has fast become one of my favorite beverages.

This is a unique drink with some technique and ceremony behind it. It took me a while to learn how to prepare it properly in the traditional style using a "cuia" (the gourd it is served in) and a bomba (a straw with a filter at the bottom).

Cuia and bomba. Image courtesy of Amazon.com

For those interested, I've included some pictures and an explanation of two ways to prepare chimarrão. The first is the traditional way with a cuia and bomba, the second way is with a French press.

Disclaimer: I am not pretending I know how to do this "right", just that this is the way that was communicated to me and it seems to work effectively. Also, I'm sure the French press method will be seen as borderline heresy to some. Your mileage will certainly vary :).

Traditional

For me, the traditional way is the most enjoyable because there's a set up that just looks neat and is fun to go through. First, you need a mate gourd or "cuia" that has a wide mouth and that tapers down to a more narrow neck before opening up to a larger bowl underneath. This taper is important for this preparation as I have discovered. If you have a more circular gourd, this process doesn't work quite as well.

The second item you will need is a bomba and the best way to describe this is it is a metal straw that has a strainer on the bottom to keep out larger mate leaf material. To finish the drink, you need chimarrão style mate leaf (chimarrão is a finely ground powder, I should add, though there are larger pieces of material included, at least in the kind I use).

We will also need hot water. That's it :).

First, take about half a cup of dry chimarrão and place it in the cuia. Put your hand over the end, turn the cuia upside-down and shake hard. This will compact the chimarrão mate leaf material towards the top of the cuia. This is important as the next step will help to hold it in place.

Next, take some water (about 1/3 - 1/2 a cup) and pour it into the cuia, then set it on its side, packed mate side down, so that the water will seep into the packed material. This will make a sort of "mate clay" that will hold together on the side of the cup. Take the bomba and use the wide end at the bottom to push the packed mate leaf to the side. This will make it so that the leaf stays compressed and provide a cavity so that water that you pour in later will not get "silted up" with fine ground mate leaf.

Then take hot water and pour it into the cavity. Let the cuia sit upright for a few minutes as the mate steeps. If you have done this right, you will be able to drink from the bomba and pull in the liquid mate without any of the gritty leaf material.

I love my daughter's description of the flavor of chimarrão. She says, "it's so clean and refreshing. It's like you are drinking a tree!" There is a definite leafy and earthy quality to it. Once you finish the liquid in the cup, just add more hot water. Keep refilling as often as desired. A cup's worth of packed chimarrão should last a person all day, though at some point the "pack" will start to break down and collapse back into the cuia. From experience, I will say it will take quite a while before that happens :).


French press

For those that don't want to go the traditional route (though seriously, I suggest you try it, it is so cool when you get the process down :) ), you can also use a French press the same way that it is used for preparing coffee (or similarly at least).

With a French press, the first step is to drop in about a 1/2 cup of chimarrão mate leaf. Add about a cup of hot water and stir. let sit for a couple of minutes. Add near-boiling hot water to the fill line and then give the contents anther stir. Take the top of the French press and put over the top to conserve heat. Let the mixture sit for five minutes.

Next, after letting the mate steep and settle, push the plunger on the French press and separate the leaf material from the liquid. The push-down should be fairly easy. If you feel much resistance before it reaches the bottom, wait several seconds to allow the contents to settle and push the plunger again.

If you will be pouring these into a cup to drink immediately, I suggest using a tea strainer to decant the liquid (it will stop the larger items but the really fine-grained chimarrão will pass through). From there, drink as you wish :).

I should also add that the French press method will also yield a strong enough liquid that you can keep adding hot water, stirring and repeating the pressing process all day if you wish. At some point, you will reach a maximum dilution but it will take a while, I can assure you :).

Aedificamus: Day One, some #HealthyTech30 Action :)

I want to say thanks to Saron Yitbarek for kicking off this idea.

She suggested on Twitter (see hashtag #HealthyTech30) that we should check-in and report on things we are doing to help us be active and eat healthy because tech workers often just sit all day.

I wanna do a challenge to be more healthy + active, especially for tech folks since our jobs tend to be pretty sedentary.

Challenge is:

1) do something active for 15 mins
2) eat/drink something healthy

Every day for 30 days. Starts tmrw. Comment if you’re in! #HealthyTech30

— Saron (@saronyitbarek) October 22, 2019

To that end, I'm happy to kick this process off with the following:

1. My "Rebounding Workstation"

This is an interesting arrangement I came up with a few months ago so that I could have a standing workstation and add some movement to it. I have been a fan of using rebounding with various video games for years but a few months ago I decided to have a look at my work arrangement and see if I could work the benefits of rebounding. For those unfamiliar with what rebounding is, it's a miniature trampoline and the person using it can do everything from gently bounce on it to more aggressive jumping and running, as well as various balancing movements. Another nice benefit of having a rebounder is that it acts as a bit of a pump for the lymphatic system and it also helps with circulation. As a person with a serious bone-break almost a decade ago, the continued gentle back and forth really helps my lower right leg even today. Finally, unlike a treadmill desk, a rebounder is lightweight and easy to move if you need the floor space for something else (working out, yoga, stretching, strap movements, etc. More on that in future posts :).

2. My Juice Bottles

Some daily food prep. In this case, we're talking juices. I have a Breville masticating juicer and I also have a subscription to Imperfect Foods. For those not familiar, Imperfect Foods (used to be called Imperfect Produce) is a service that sells "ugly food", or in other words, food that would not be deemed "sellable" due to cosmetic issues. Perfectly good food, just not "pretty" and I'm totally OK with buying produce that may not exactly square with grocery store pretty standards. After I cut and prep everything, it really doesn't matter as long as it tastes good.

OK, so the second step to this is that I have a bunch of handled Mason jars with screw tops and straws. Each week when I get the bulk of my produce I take a bunch of it and I juice it. The resulting juice is super pulpy (which I like) and this means I am ingesting a good amount of the fiber as well. There is, of course, the leftover spent fiber that I typically freeze and use in other dishes. Waste not want not ;). I then put these juices into the jars and keep them in the fridge. I grab them for days when I'm not super motivated to cook a bunch and I just want to grab something and have it ready to go. In the mix today are:

- carrot and ginger juice (that's the orange one)

- kale, spinach, and zucchini (that's the green one)

- yerba mate made "terere" style (that the tall, dark one. I make that with a French Press, more on that another day :) ).

Tomorrow I will talk about some home exercise stuff, and maybe share some food options you can make with pomace from juicing :).

Two Down, Eight More to Go - Tackling More "30 Days of Testing" Challenges

As a re-enactor, a performer, and a musician, I appreciate the fact that there is a need for regular practice in any endeavor.

- While I dress up like a pirate and participate in occasional stage shows, I need to actually practice swordwork so that I can be prepared and ready, as well as SAFE, during stage performances. In short, I need to keep my body in practice with rudiments and fencing drills.

- As a musician, I can't just show up and improvise (well, I can and I have and the results have been predictably embarrassing). To be able to play at any decent proficiency and dexterity, I must practice, even if the practice I do is to play things by ear so that I can do better live improvisation. The same goes for writing songs. If I want to write better songs, I have to (gasp!) write songs in the first place. It's a little silly to think I'm going to get inspiration and write the perfect thing every time. Likewise, if I use the music theory I do know and write songs with it, I may not make something brilliant every time but my odds of writing something good go way up. Much better than if I just wait for inspiration to strike.

- When I make clothes for historical garb or cosplay, I can't just expect to come in and knock everything out the first time in perfect order. I'm just not that skilled a tailor. I can, however, make mocks and practice and try out the ideas so I can get it solid enough to make the items well.

Why should I think that as a blogger and as a tester I am just going to have intelligent things fall into my lap? The answer is "things probably won't but they definitely won't if I don't practice or prepare for them.

This brings me back to the "30 Days" Challenges. For various reasons I looked at a number of them and said "oh, that would be cool, I will check that out later" or "hmmm, not quite in my wheelhouse, I may check that out further down the road." Any guesses how many of them I've come back to? Yep, I've not come back to any of them except for the two that I chose to hit immediately. Notice that those both completed and I learned a lot from both of them. Let's have a look at a little graphic:


There are ten challenges there. Two are done, eight I've never started. Well, that's going to change. Next up is "30 Days of Automation in Testing". Why? I'm in the middle of learning how to set up C# and .NET Core for automation needs.

The problem is, we're already up to the 16th of October. Not a really convenient start time, right? Old me would say "OK, I'll start this beginning of November" and then I'd forget about doing it. I'd still feel good because I told the world I'd do it. I mean, who is going to check up on me, right? Well, that's a lame attitude and the answer is I'M GOING TO CHECK UP ON ME!!! 

By the way, expect me to talk about "Writing While ADHD" but I'm not going to promise a timeline for it just yet ;).

So what's my plan for the "30 Days of Automation in Testing"? Simple, I'm starting it today. Seems two posts a day should be enough to get me back on track and cover 30 days (that may be aggressive and ambitious but hey, fools rush in where consultants fear to tread ;) ).

Am I Really So Ordinary? - a #PNSQC2019 Blog Retro

This has been an amazing few days. I've received a presentation award from my peers here. You all voted with your evaluations and you felt my score warranted the second most highly rated presentation of the conference. WOW!

I'm humbled by this but I'm also a little embarrassed. Why would I be embarrassed? Because for the past five months, my blog has been quiet. Why is that? Because I've felt that I don't have anything important left to say. TESTHEAD has been on the air for almost ten years. There are over 1200 blog posts I have written. What more could I possibly say without repeating myself? What can I possibly add that would be even remotely interesting?


I don't know if anyone else has these thoughts from time to time... or often... or every single day... but yeah, I do. I had a great conversation last night with a fellow presenter (they may or may not be cool with me sharing this so I'm cloaking in a little anonymity... but I'm pretty sure anyone who knows us can guess ;) ). As I was talking about how I struggled to come up with an idea this year and that I wondered if my experience would even be all that interesting, we recapped a few things and thoughts:

- experiences are all we can really share and there what people actually relate to. Me setting myself on high and offering pronouncements is boring. Me telling how I got completely lost or frustrated with a situation and what I learned from it is much more valuable.

- I joked that so much of my talk was "blinding flashes of the obvious" and the response back was "was it really? If it was so obvious, why was it a revelation when you addressed it?" Point being, what may seem patently obvious in hindsight may be hidden or not understood by everyone else. In short, if you are confused, it's a good bet a lot of other people are, too.

- it takes a lot to get people to get up on a stage or in front of a group of people to be willing to speak. What we may see as banal and every day is a major step out of the comfort zone for 95% of people. The act of presenting is courageous in and of itself, much less someone willing to do it again and again, year after year.

- what's more, think about what people do to agree to come to a conference in general. They give up their time, their families, their work commitments, their home commitments, many of them pack themselves into a plane for several hours and are not at all thrilled about the experience, yet they go because they want to hear what might give them an edge, a new idea, a new angle to help them do better work every day. They want to hear what you have to say, and really, the only worthwhile thing that you can share is your own experiences.

I should also mention that the thoughts for my talk didn't come together fully formed. The paper I submitted went through three revisions and extensive feedback from two other individuals that helped me take ideas that were half baked and get them to make more sense, as well as to be able to step back and help me emphasize the areas that needed to be and push back or disregard areas that didn't add as much as the parts they suggested I emphasize. For those who voted for my presentation, I must be absolutely clear that "I HAD HELP!"

Others have asked if I will be back next year and what I might talk about? The answers are "likely, yes!" and "I really do not know at this stage" but I have a few ideas. One thing I want to do is go back and review the other "30 Days" challenges that the Ministry of Testing has put together. I have several areas in my own work environment that is requiring me to step out of my comfort zone (have I mentioned I'm trading in my MacBookPro for a Windows 10 machine? Have I mentioned that I'm looking into what it takes to program in C# and run on .NET Core? Yeah, those are new realities for me. If you asked me last week, I might have said "yeah, no one is really interested in that." Today? I have a totally different opinion on that front. I'm still learning things and there's a lot to learn so it only seems reasonable I keep learning in public the way I've said I would :).

PS, I've been on a voyage of musical discovery with my younger daughter recently and part of that has been to introduce singer/songwriter Paula Cole to her. Today's title borrows from her first single "I Am So Ordinary" so credit where credit is due ;).

A Show of Hands - a #PNSQC2019 Live Blog

Today is the workshop day at PNSQC. I'm the moderator for Melissa Tondi's workshop on "Efficient Testing". As the workshops are an add on and paid for by the attendees, out of respect for that I do not live blog workshop goings-on; If you want to take part, come out and sign up to be a part of it ;).

Instead, I' like to talk a little bit about what I think really makes PNSQC unique, and that is its emphasis on working with volunteers.

• If you submitted a paper and you received feedback, that person providing feedback is a volunteer.
• If you interact with the web site, those updates are done by volunteers.
• The registration, room monitoring, moderating of tracks, etc. are done by volunteers.

In short, this conference has so many opportunities to volunteer and participate. Many of the opportunities available will get a person a free ticket to the conference. Volunteering for workshops also gives a person the opportunity to participate in that workshop for free. While there is no guarantee that a person will be able to moderate or facilitate the specific workshop they want to participate in, odds are still pretty good that if you show interest early, you can moderate your first choice.

The bottom line here is that the conference is an excellent one, IMO, and the volunteers go a long way in helping foster that experience. 

Testing AI and Bias - a #PNSQC2019 Live Blog

Wow, have we really gotten to this point already? We're down to the last formal talk, the last Keynote. These conferences seem to go faster and faster every time I come out to play.

I've had the chance to hear Jason Arbon talk a number of times at a variety of conferences over the past several years and Jason likes to tackle a variety of topics with ML and AI. Thus, I'm definitely interested to see where Jason was going to go with the topic of AI and Bias. This is a wild new area of testing and as many of you all know I am fond of Carina Zona's talk "Consequences of an Insightful Algorithm".

OK, we understand bias is there but what can we actually do about it? Well, here's the spoiler. You can't remove bias from AI. That is by design. AI takes training data and based on the training system it learns. Literally at the start of the process bias enters in. The point is not to eliminate bias, it is to make sure that undesirable bias is not present or is minimized.

Think of a search engine. How can a machine look at a number of source articles and then, based on a query, decide what might be the most important information? We start with an initialized system. Think of it as a "fresh brain" and not in the zombie sense ;). From there, we then go to a training system, which is information already graded and scored by a human or group of humans, so that the system can train on that data and those values. Can you guess where the bias has crept in? Yep, it's there with the training set. If the system guesses wrong, it gets negative reinforcement. If it gets it right, it gets positive reinforcement (from a machine's sense of reward, I guess).

There are other factors at play such as commerce (ads will get preference or at least money will). Crawlers start on "popular" sites and then look at less linked sites. It will also be biased, by language, education and very likely dominant gender and race. There is also the fact that Microsoft Bing is set by default for Windows. For those people who don't know how to change their search engine, you end up with a "lowest-common-denominator" of users that match up with some weird demographics. According to Jason, Microsoft Bing's core user group is single women in the midwest (he said it, not me :p ). Indicative of that is that much of Microsoft Bing's largest search volume comes from that demographic. What might that indicate what is "feeding the neural network"?

There is also Temporal Bias or "Drift". Over time, the searches can be effective by many things such as weather, politics, astrology, etc. Sample Size can also affect the way that data is represented. One way to check this is to keep changing the sample size until the scores stop changing. Not a guarantee but at least it gives a better feeling that more people are being represented.

There is also a bias in the literal training data. In most cases, the people who provide seach data training sets are for people paid $20/hr or less. In short, the people feeding are neural networks are not those of us who are designing them. We can debate if that is a good or bad thing or if software engineers would be any better.

There's even a Cleaning Bias. Bing cleaned out mis-spellings, random numbers and letters, etc.and the irony was that Google didn't do that and thus Google can even help people find ht they are looking for even if they misspell it.

What happens when there is no "right" answer? Right answer meaning there is one answer for a particular word and there are multiple possible different options for the same word?

As Jason has said a few times, "are you all scared yet?!" Truthfully, I'm not scared but I am a lot more cynical. Frankly, I don't consider that a bad outcome. I think it helps to have a healthy skepticism of these types of systems. We as testers should remain relevant a bit longer if we continue to be ;).

Testing in the Golden Age of Quality - a #PNSQC2019 Live Blog

Jennie Bramble and I have joked a couple of times today that with her giving two talks (one totally impromptu because a speaker went MIA) as well as running the Panel Discussion for
"Testing in the Golden Age of Quality: Where are we and where are we going?", we should just change the working name of PNSQC 2019 to "The Jennie Bramble Show" ;).

Anyway, we have a wonderful panel including  Carol Oliver, Nermina Avdic, and Mallory Brame. We are chatting about "the world we live in and life in general" (and bless you if you actually get that reference, though it may indicate you are at least my age ;) ). Quality is at the forefront of software decisions now more than it ever has been. There have been huge strides in tools, methodologies, and areas that software testers and testing can get involved with. Not bad for a career that was considered "dead" ten years ago (and yes, that "dead" is in huge air quotes and I get the original context, don't get at me (LOL!) ).

Some areas that have been discussed:

- How has the proliferation of tools and paradigms helped the software testing landscape (answer: it has, a lot!)

- What sort of skill sets might we look for in someone interested in security testing? It starts with the code review and then we need to be able to consider attack vectors based on the code used. Security Certificates like CISSP are also helpful but not easy to get.

- How do we show the best value to our clients and how has that changed? By helping move the process along, by helping test early, identifying paths that would be good automation candidates, showing that early testing makes for easier to fix bugs and that pairing can possibly prevent bugs in the first place, thinking of areas developers wouldn't have considered to test.

- Advice for how to communicate with devs to take bugs personally? Have a conversation around the proof of the issue, try to find common ground, it's possible both groups have different understandings,  realize that pushback is natural but being able to support with evidence goes a long way, don't think in terms of "I'm right, you're wrong, shut up!". Unless you're Eugene Lee Yang, then, by all means, go ahead ;).

- How does your company treat bugs such as bugs in production vs. bugs in regression? If the bug is not easily discoverable, we may defer but generally, we will do what we can to fix them. Customer anger may tip the scales ;). Ask the question "so, what would happen if a customer actually did find this?"

- In this Golden Age are we still numbers-driven or is that changing? Jennie steps out of moderator role to say that numbers are a terrible rubric for validating quality because numbers can be gamed so easily as to be effectively meaningless. Fewer numbers, more morale-driven and skills-driven. The numbers will still be present but expect to see we are moving away from it.


Questions that didn't get covered. Any chance readers might want to follow on in the comments :)? If so, sound off!!!

----------

What are the practices for high-value test automation?

What makes automation valuable in the right now and what will it help us achieve in the future?

What methodologies are most prominent right now?

Are we doing more exploratory or ad hoc testing as opposed to test cases and scripts?

Where does QA fit into the life cycle?

How can we move around the SDLC and create value in new places?

What are ways that companies are thinking quality first and what can we do to influence that?

Where do you see QA/automation moving in the future and do you like what you see?

How can we make an impact on the future, share ideas and come together as a discipline?

Should we bleed more into development, for example, testers fixing defects, and is this something you do at your job or would embrace?

How can we make the golden age of quality last?

A11Y Testing Using an Intelligent Agent - a #PNSQC2019 Live Blog

All right, we are in my wheelhouse now :).


As an #a11y advocate, I spend a lot of time talking about and hoping to get people excited about and see the value in focusing on the benefits of thinking about Accessibility.

First and foremost, let's talk about a sobering number. 75,000,000 people need wheelchairs but cannot afford them. Why is this number important? It underscores the moral obligation that we as a society have to help these people who otherwise would be left out of any realistic operation in society. The World Bank estimates that 1.125 billion people deal with some significant difficulties in daily life due to a disability. That is 15% of the world's population as of now.

These numbers, I hope, give emphasis to how many people are affected by Accessibility issues. There are also news stories that time and time again that show that businesses are slowly waking up to the fact that, if they don't do o out of moral or financial obligation, they may well end up paying for it in legal fees and lawsuits.

When I test for Accessibility issues, I tend to use the Web Content Accessibility Guidelines (WCAG) produced by the World Wide Web Consortium (W3C). Sounds like a mouthful but ultimately it comes down to:

Is a site Perceivable?
Is a site Operable?
Is a site Understandable?
Is a site Robust?

In other words, does your site "POUR" ;)? No, that's not really a thing but I laugh about it anyway.

All right, enough merriment, how does Keith recommend we actually test and what can the automated tools actually help with? On the whole, automated testing has a LONG way to go when it comes to addressing Critical and Cognitive issues with sites. Most of the issues found have been found with human discernment (for those who have followed my comments about Accessibility over the past few years know that this meshes with my general opinion. It's nice to see actual data points that support it, too ;) ).

O course, with the title of this talk, I'm expecting that Kevin has some sort of a software anwer to this dilemma. To that effect, let's have a look at Agent A11Y!

Agent A11Y is capable of semi-autonomously exploring a website and evaluating its compliance with WCAG guidelines. As far as automated tools are concerned, that's a big step.

There is also additional tooling around manual testing of WCAG requirements, as a lot of WCAG is difficult to fully automate.

I had a hand in helping review this paper so I have had some experience with the end results of today's presentation. Having said that, I'm very excited to get a chance to play with this in the wild.

Agile Where Agile Fears To Tread! - a #PNSQC2019 Live Blog

Thomas Cagley is a fun guy to listen to in just about any situation. He's been a guest on The Testing Show and I've listened to his own SPAMcast podcast (SPAM in this case means "Software Process and Measurement" ;) ) but this is the first time I've actually listened to him speak.

Tom opens up his conversation with a word I will not even try to write out much less pronounce but it has to do with Germany's Strict Beer Purity Law that has existed since the Middle Ages in Bavaria. He then mentions that his wife is gluten intolerant and thus, because of that she cannot drink beer. She can, however, drink something referred to as a "tea beer". Is it "beer" in the classic sense related to the German Brew Law? No. Does that render the beverage irrelevant or not enjoyable? Not in the slightest :). I can't speak for the beverage in question and the idea of a "tea beer" absolutely intrigues me (I don't drink but I do find the concept fascinating).


The key point to this is that we are back to the lower case versus upper case "[aA]gile". As Dawn Haynes suggested that we should tear down the Agile Temple (granted, those are more my words than hers), Tom is discussing the idea that we can take elements of Agile that are helpful and we can use them in the places we want to without necessarily committing to a full-blown "Religion of Agile"

The goal of "agility" is to move more nimbly, to be quicker, to not have to do such heavy lifts, and be able to get the product out to customers faster and more frequently, with less risk. News flash, you do not have to adopt every element of the Holy Eucharist of the Agile Temple (wow, sorry if that sounds a little harsh but I'm picking the words for the vividness, not for the snark... well, not maybe a little bit for the snark). Basically, we're back to the Beer Law. If it's not [ABCDEFGHIJK] then it's not Agile. Well, OK, maybe we can't do [ABCDEFGHIJK] but what if we were able to do [AEFGJK]. Do we just throw up our hands and say "nope, can't do just those, that wouldn't be Agile". Sadly, there are people who do say exactly that. I'm a fairly polite fellow and as such I will not call these people what they richly deserve to be called (hint: it rhymes with "boron").

As smaller components are implemented and benefits are seen what is likely to happen? The other elements will either fall into place as a matter of course or they can be introduced as needed and improved upon as the team gets better at what they do.

Creating Quality with Mob Programming - a #PNSQC2019 Live Blog

Thomas Desmond has helped me get my head around an example of something I've been interested in but haven't actually been able to actively participate in... what does Mob Programming actually look like?

I understand it as a concept but truth be told, programming in my organization beyond a pair arrangement is... challenging. The biggest challenge is the fact that we are all distributed. We've tried programming as a group via Hangouts or using tmux but again, it's a challenge to get that done with more than to people. Thomas is showing how is organization sets up these massive systems with multiple big screens, with multiple keyboards on rolling desks that can go anywhere in the office. The key idea here is that all of the people (optimized to four) are in the same place, at the same time, talking together simultaneously, and all interacting on the same computer.

Thomas is describing a situation where they mob program on all production code. As in, they don't have their own desks. They work as a group on a single task; designing, coding, testing, and releasing all together. The thought of being able to do this all day, every day, on all projects both seems cool and a little weird. In the neat way, not the unnerving way.

A tool that they use is called "Mobster" (neat, need to look this up) that has limites on who can be the driver (IOW, hands on the keyboard) and who can be the navigator (guiding direction but not necessarily behind the wheel) at any given time. The goal is that the roles switch and everyone gets their turn. For an idea to be implemented one of the navigators must be able to explain the idea clearly so that the driver can implement the idea. Ideally, everything is explained, everyone else in the group can hear the idea(s), and they can comment on the idea before it actually gets implemented.

I have struggled with where I would be effective as a tester in a Mob Programming environment and now that I have seen it explained as implemented by Hunter Industries. They actually throw new people right into the mix. Counter-intuitively, they come up to speed faster this way than they would if they were to be trying to get up to speed in a traditional development environment.

Thomas emphasizes that the benefits of mob programming are:

- live code reviews
- sharing knowledge
- greater idea sharing
- fewer meetings
- more engagement
- increased code experimentation

I must confess that this is a lot more tangible an idea now and it makes me excited to see if/how we might be able to implement it. Any thoughts on how to mob while fully distributed, let me know, please :).

Cutting Releases Cadence - a #PNSQC2019 Live Blog

OK, time to put on my Release Manager and Build Manager hat. For the past few years, outside of being a software tester this has been my most visible functionality within the team. There are a lot of moving parts in this process that I had to come to grips with and get a feel for what exactly it was that I was doing to make releases and deploy them. We do well with Continuous Integration. Continuous Delivery and Deployment are areas we can certainly do better, hence why I am here :).

We have some rules in place at my company and its parental units that mean that true push-button Continuous Delivery and Deployment will likely never happen in actual production. Well, saying "never" may be a bit of hyperbole but much would need to change before our organization would be OK with doing it that way. Still, just because there are limitations to CI/CD, that doesn't mean that in other cases we couldn't or shouldn't be able to do it. We have development environments, staging environments, and integration environments. They need to be provisioned and set up just like any customer site. Those steps are not exactly changing day to day if you get my drift :). Thus, it makes perfect sense to think that we should be able to do CI/CD on a more frequent basis, even if we are the only ones (the engineering team) who reap the everyday benefits.

I can totally feel how And Peterson's organization went through the processes he did to try to wrangle this monster to get a system in place that required less hand-holding and allowed for more time to work on genuinely interesting challenges.

Also, just because you have a process that is push-button does not mean that you always have to do it that way. All that it means is hat the parameters necessary are well understood and repeatable. If you can repeat them, you can standardize them. If you can standardize them, you can package them. If you can package them, you can set them into containers or other structures that allow us to maximize the amount of information that replicates and doesn't change, speeding up our deployments and limiting the time we have to wait between a release build finishing and th time when an environment is up and running with our application in a usable state.

Even with this approach, we are still limited to other teams in our company and what they can and will be able to release. Again, just because your product may not go out every day, there is no reason to not be able to create a staging environment that will benefit from these changes. While we may have a quarterly release cadence, there is nothing stopping us from getting into a daily cadence to push features in fully qualified builds to our staging server. Granted, this does mean that we have to go back and do a little bit of repeating to see if pushing a lot of the changes to a numbered build introduces anything unusual. Still, we have had a chance to see everything working in the staging environment so this shouldn't be a barrier in practice. I say that now, but let's see how that works in practice ;).

Being More Agile Without Doing Agile - a #PNSQC2019 Live Blog

Can I share a possibly unpopular opinion? I am not a fan of "Agile".

Now wait, let me clarify. I love BEING agile. Heck, who doesn't? I don't have a problem with the adjective. I have a problem with the Noun.

Also a confession. I'm here mainly because Dawn Haynes is talking. I've known Dawn for years and the irony is that I have had precious few times that I have actually been able to hear Dawn speak. Thus I consider this a perfect blend of opportunity and attitude :).

I like "little a" agility. Again, the actions and abilities. Those are all good things. They are helpful and necessary.  I like being nimble and quick where I can be.

What I have found less appealing in "Big A" Agile. Mainly because I find that when organizations try to implement "Big A" Agile, they become anything but "little a" agile.

As a software tester, I have often found that there is an afterthought when it comes to testing in Agile implementations. More times than not, what results are teams that kind of, sort of, maybe do some Agile stuff and then retrofit everything that doesn't actually feel right into a safer space.

Dawn emphasizes that the best way to achieve the goals of "Agile" is to actually "be agile". In other words, forget the process (for a moment) and focus on yourself and what you’re trying to accomplish.

A Comfort Zone is a Beautiful Place but Nothing Ever Grows There

For teams to get better, they have to be willing to go to places they don't really want to go to. There is a fear that going into the unknown will slow us down, will send us down paths we are not thrilled about going down, may not even get us to our end goal quickly. So we put a lot of emphasis on what I call "the priestly caste" and "the temple incantations". I'm not trying to be flip here, I'm saying that there are a lot of rituals that we reach for when we are not 100% sure about what we are or should be doing. As long as the rituals are met, we see comfort there, even if the rituals are adding little to no actual benefit. Are retrospectives helpful? They can be if they are acted upon. If they aren't then it is an empty ritual. Granted, it may take time and commitment to see the results of the retrospective findings and real results may not be manifest for weeks or months. Still, if we do not see that there are actual improvements coming from those retros, what is the point of doing them?

One of the interesting developments on my team related to agility and moving more quickly and effectively was to allow myself to wear whatever hat was needed at the moment. I'm not just a tester. Some days I'm a part-time ops person. Some days I'm a build manager. Some days I'm a release manager. Some days I've been a Scrum Master (and, in fact, I was a dedicated Scrum Master for three months). I was still a tester but I did what was needed for the moment and often that meant not being a "Tester" but always being a "tester"... see what I did there ;)?

Are test cases necessary? It depends on what you define as a test case. In my world, I go through a few waves of test case development. Almost never do I start with some super detailed test case. Typically I start with a 5000-foot view and then I look to get an idea of what is in that space. I may or may not even have a clear idea of how to do what I need to do, but I will figure it out. It's the process of that learning that helps me flesh out the ideas needed to test. Do I need to automate steps? Sure, but generally speaking, once I automate them, if I've done it right, that's the last time I need to really care about that level of granularity. Do I really care if I know exactly every step necessary to complete a workflow down to the property IDs needed to reference the elements? No, not really. Do I need to know that a unique ID name exists and can be used? Yes, I care a lot about that. In fact, that's about the most important finding we can make (see my talk about "Is this Testable?" about more of my feelings on this :) ).

The key takeaway, care more about the work and about being nimble than bowing to the altar of AGILE. I find much to value in that :).

Remember This! - a #PNSQC2019 Live Blog

Day Two is underway! Before I start, I want to say thank you so very much to Erin Dees for inviting me and a group of people out to Mother's Bistro for what turned out to be an evening of great food, hilarious conversation and me getting back to my hotel room and just deciding I was done for the day. I like when that happens :).

Today's first keynote talk is being given by Scott Crabtree. Scott is the founder of Happy Brain Science and his message for today is
"Constant change means constant learning".

Scott's opening monologue was an amazing dream sequence and an interesting phenomenon. Have you ever noticed that there is a break ith your immediate memory when you wal through a doorway? I've certainly noticed this every time I try to remember something I need to be doing.

Scott is going to lead a "memory palace" game, so I am going to literally sketch this idea:

- Come up with a space that has twelve distinct areas to remember (interestingly enough, my house with its rooms, halls, and garage works for this :) ).

- Go into all of the spaces one by one and imaging them empty.

- In the first place, I put berries (master bedroom has berries, the brain is in my body, berries boost memory. neat :)!!!). Lots of berries, make it weird. I'm picturing a ship made out of berries because of pirates :p.

- In the second space, put meditation there. That's my son's room, or what up until a few years ago was his room. It's aquamarine in color, so this is easy for me to envision. Scott says get weird, so I'll picture my son in his over the top hip hop gear sitting in the lotus position. Actually, I'm not sure that's all that weird, that just kind of sounds like my son (LOL!). For the record, I do see a value in meditation and I do it, not as frequently as I probably should.

- Repetition causes learning. Brains are wired to remember visuals than raw facts. Repeat to teach. In the third room, create repetition. Scott recommended a record skipping. For me, I prefer a looping rhythm and melody. There' a melody that is famously called "The Riff" that I can picture looping as a great sense of repetition. The third spot is my downstairs bathroom, so in some ways, this is really appropriate.

- The fourth location in testing. Ironically, the fourth location is my daughter's bedroom and they are both recent high school graduates so I can think of them both taking tests and testing limits. My daughter recently did her driving tests so hey, it works :).

- The fifth place, place a smell. I absolutely agree that smell can intensely focus on thinking, both pleasant and unpleasant. I keep an essential oil diffuser in my office and I use it regularly. I don't place any "woo" in the value of essential oils, to be clear, but I do find that intense smells to be focusing so I use them. Scott recommends Rosemary and while it is good, I find that a mix of Lemongrass and Tea Tree Oil to be the most effective because together they are distinct and fairly pungent. This is my hallway that connects these previous four rooms.

- The sixth place is focused on picturing memories. This would actually normally be my hallway but since I've used that already, lt's go with our front room/dining room combo. It has a fair amount of decor, some pictures, some curios, but it's a definite visual room so this works :).

- The seventh spot is emotional. THAT IS MY KITCHEN (LOL!). I love to cook, especially stuff that is uncommon and unique. Some of my cooking adventures have been sublime, some of them have been terrifying, but cooking is a peak of emotion. The kitchen is my emotional center and I can work with that! Emotion is me throwing food around from pan to pan over a set of running burners. It may not work for anyone else but it works for me.

- The eighth spot is my stairway that leads upstairs, and Scott is recommending exercise. this is easy because the one exercise thing I tend to struggle with exercise-wise is climbing floors. I have a set number I try to get to each day and some days I succeed and some days I don't. Thus stairs work for me.

- The ninth spot is space or spacing out learning. this space is our family room upstairs. It's an all-purpose room with a lot going on so it works as a space-out place. It's also where my daughter draws and does a lot of her artwork so that is easy to picture :).

- Tenth spot. Look left and right to find a memory. The upstairs bathroom has a weird configuration with two carriage lights and a mirror in the center. It sort of looks like a face so left and right looking works for this room :). Scott recommends eyes, and in this case, the carriage lamps work for eyes. Neat!

- Eleventh Spot. The doorway effect. This represents my office and I can honestly say that more times than not the doorway effect has happened as I leave this room more than any other. My office is already weird as it represents everything I do as far as picturing things, I already have a memorable image, it has a poster of the Attack on Titan protagonists (Eren, Mikasa, and Armin). I'll mentally picture the flying image as the whole door.

- Twelfth and Last Spot. End with the most important information. There's just one place left in my house and ironically it's where I figured it needs to be the last spot. It's my garage. Yes, we put our car in the garage (and amazingly, we can actually do that in our San Francisco Peninsula neighborhood. Thus I picture END being my garage. The way I represent that is with an eight-foot closet rod. Why this? It's where all of my re-enactment creations get hung and stored and categorized (oh, now there' a topic ;) ). It's the end result of a lot of work and creativity so it's easy to remember END in this capacity.

So what is this actually? It's called a "memory palace" and it's a way to help us to keep memories vivid and intact. If you walk through these steps, my guess is that you will likewise sharpen your memory. This looks like something I can actively play with when I get home :).

Surviving The AI Testing Apocalypse - a #PNSQC2019 Live Blog

Here's an interesting thought.

Let's say we are building a road. We come up to an anthill. What do we do? the answer is pretty simple. We run through or remove the anthill.

Now let's rephrase that... what if we develop AI that goes to a point of surpassing human civilizations barriers. What if we set AI on a goal that we get in the way of. Will AI look at us as little more than an anthill?

Unsettling? Maybe. Accurate? Remain to be seen, but again, maybe.

Now do understand, I have not cut my throat yet or anything. I have children and I want to see them grow up and thrive so I have a vested interest in a future for humanity. Yet I would be lying if I didn't say that I look at machine learning and AI development with both a sense of trepidation but also a sense of excitement and wonder.

Now let's step back a bit and let's talk about AI as it relates to software testing. Will AI and ML devastate software testing? It's possible but at what level? Do we believe that it can replace human creativity? Could a computer AI figure out how to write a coherent symphony? It can make some straightforward musical progressions now. Are they terribly exciting or unique? No, but they do follow the fundamental rules of music theory and to many ears, there's not really a difference. The AI mak serviceable and competent music. I'll reserve my comments about how creative or unique the music created is. Then again, I could say the same about most of the music humans are creating at the moment, including me.

We can certainly see AI/ML interacting with video games and improving/learning how to play a game. It's interesting to view in real-time.

This is all a premise and y thoughts on Dionny Santiago's talk about "Surviving the AI Testing Apocalypse". Bold statements but let's look at a few words.

Apocalypse is a much-maligned word. It sounds automatically like it is a calamity. However, Apocalypse comes from the Greek word "apokalupsis", which means "to uncover". Thus, do I think that we face an uncovering of AI and interacting with it? Yes, I absolutely do. BTW, it also means "to reveal" which is where the word "Revelation" comes from. I feel bad for Greek, it's getting a bad rap. Why does Apocalypse get the bad press while Revelation is so much more hopeful (Eschatological meanings not being included here).

AI is here. It's real. It's happening now. It will keep happening. It's important and we are benefitting from it. However, I do not believe we are facing anything like The Matrix or The Singularity. I likewise also believe that human invention and creativity will win out in the end. That doesn't necessarily mean that *I* will win out. I may may well be a casualty. Nevertheless, what a time to be alive :).

The Release Day Crime Scene - a #PNSQC2019 Live Blog

Bill Opsal and I have been in talks over the past year or so about putting together a simulation game that we jokingly called "CSI: Release Day". Think Clue but for software teams.

The premise is as follows.

There are six roles: Release Manager, Dev Lead, QA Manager, DBA, Project Manager, and Customer.

Each role has three key pieces of "mandated transparency", meaning if you have a role, depending on the scenario being run, you can say anything about what your role is but if you are asked a question where the answer is your piece of transparency, you have to say it. If your piece of transparency also indicates that you are the guilty party, you likewise have to say it.

The goal is to have the Crime Scene Investigation team question the people with their roles and see if they can ferret out the problem and the reason why it caused the problem.

All in the name of truth, justice, and the releasable way.

Bill and I enjoyed talking about this over the past few months leading up to this so it was fun to finally, FINALLY get to present it. On the whole, it seems like a lot of people enjoyed it and it also seems like the game has legs, so it's entirely possible we may expand on this and lmae ita little more concrete for a future conference :).

Test Scenario Design Models - a #PNSQC2019 Live Blog

This year I decided to emphasize the Test Engineering track, and I'm not saying that just because my talk was part of that track. It is actually one of the key areas I would like to see some personal improvements in what both I and my company do. With that in mind, I decided I want to check out "Test Scenario Design Models: What Are They and Why Are They Your Key to Agile Quality Success?"

Some interesting perspectives:

87% of respondents say management is on board with automated testing
72-76% of respondents say they are doing test automation and scripting
Most organizations are between 0-30% automated.
Of those, about 40% of their tests are completely redundant, meaning they are really not worth anything.

There seems to be a disconnect here. I do not doubt it in the slightest.

Systems are more complex, the proliferation of environments is continuing, manual testing will never be able to cover it all, and the automation we are doing is not even helping us tread water effectively.

Can I get a "HALLELUJIAH!", people?!!

What could happen if we actually said, each sprint, "we are going to send a couple of hours to actually get our testing strategy aligned and effective"? Robert Gormley encourages us to say "oh yeah, we dare!" :)


This is where the idea behind Test Scenario Design Models come in.

The goals are:
- user behavior is king
- test cases are short, concise, and business language-driven.

We should not care if we have 4500 total test cases. What we should care about is that we have 300 really useful high-quality tests. The numbers aren't really relevant the point is to have tests that are effective and to stop chasing quantity as to any kind of a meaningful metric.

So how do we get to that magical-unicorn-filled land of supremely valuable tests?

First, we want to get to a point where we can be specific with the tests that we need to run but no more specific. Extensive test scenarios are not necessary and should not be encouraged. Additionally, we need to be emphasizing that we are looking at testing relevant to the User Acceptance Testing level. That's where we find the bugs, so if we can push that discovery further back, we can work on more important things.

Why We Need New Software Testing Technologies - a #PNSQC2019 Live Blog

Dr. Carol Oliver makes the case that we are now seeing a proliferation of environments where software can run. Once upon a time, programming was a bespoke profession. Computers were standalone and in many cases not interoperable. That has certainly not been the case in the past few decades. Mobile and IoT software have the added challenge of even more potential places and environments for software to run. The testing paradigms that we use, however, have not kept pace with the proliferation of platforms. This should probably not be a big surprise but it is a real issue.


Dr. Oliver has her paper to describe a Release-Readiness Levels Framework with vocabulary and structure so as to discuss what software testers would like to be able to test and what the existing tools and technologies enable them to test.

Mobile and IoT show how many potential devices exist and how very few of them we actually test with. We are especially not looking at all of the possible variations of hardware and software. We literally can't. Modeling software also uses such a large level of abstraction that it is very narrow the range we are actually testing vs. what is out there.

Areas that can be addressed and looked at for pros and cons are:

• Physical Devices (the actual proliferation of machines)
• Virtual Devices (abstractions to represent real machines)
• Simulation Environments (even greater levels of abstraction, one size fits most, theoretically)
• Visual GUI Tree (what prt of the GUI can you interact with? How much can you interact with it?
• Image Comparison (How do we determine we have the appropriate technology to display the images correctly and appropriately.
• Code Manipulation
• System Monitoring

One of the things you will notice is that most of these areas are not immediate or easy to test. Additionally, many of these areas require a technical bent to the testers in question. We are not even close to having general-purpose tools but we do have some ideas as to what those general tools would entail. Needless to say, I am intrigued and will read the relative paper more closely :).