Summing Up "The Ludicrous Year" that was 2020

 I had a feeling that my run of the joke that was the lyrics of  "Once In A Lifetime" might come to an end in 2020 after ten years but frankly, nothing in those lines can be used that can sum up this year that absolutely redefined reality for so many. The only line that matches would be a repeat anyway... "well, how did I get here?"

It started out with my looking forward to a lot of new opportunities. My band, Ensign Red, had just recruited a new guitar player to replace our longtime outgoing guitarist. We were looking forward to a year of a new infusion of ideas, of songs, of shows, and other avenues to perform. I was just coming off of a string of dates with the Sea Dogs/Paddy West School of Seamanship during the Dickens Faire and I was looking forward to a year of performances where I'd be able to show versatility and get to perform in more places. 

Then "The Ludicrous Year" took over.

For four whole months, we didn't do anything together as we were under "shelter in place" rules and we did our best to follow them. we invested in Zoom setups. We invested in a shared file store to trade song ideas. We invested in new interface technology to capture what we wanted to record and make it as close to seamless as we could so everyone could work on ideas. We put together a short series of interviews with the band members and we would put some rehearsal footage together to leak out to people to hear what we were doing. I'd Zoom with the members of the Sea Dogs and Paddy West to stay in the loop and hope there might be some performances of some kind (spoiler, nope, there weren't). Still, I feel like I got the chance to get to know the other performers in a way I might not have otherwise and I had the chance to get a better feel for where I might be able to make contributions in some way in the future, whenever that future will be and what it will be.

I started out with a team that I had worked with for seven years, a product I knew well, and a pretty clear path as to what I had to do to keep it going as we expected. 

Then "The Ludicrous Year" took over.

With the need to reposition and reapportion who was working on what, including the tough decision to let go of most contractors,  I was asked if I'd be willing to join a group that needed a tester and hadn't had one before, to go into an avenue of our company that was utterly foreign to everything I had been working on up to that time. There were no familiar faces, no familiar rituals, no familiar workflows to fall back on. I said yes and thus started my learning about a whole new way of looking at testing and also looking at data. I took on an automation project unlike any I had previously worked on. Far less UI focus, much more API manipulation. Less browser interaction, more flat file manipulation. Less "does this look right?" and more "is the data accurate?" challenging, but interesting. I had to say "goodbye" in a way to a long time family but I got to meet a whole new team. The challenge was of course that I struggled to keep up with the people who were my long-time team as we were now focused on different things and my old comfort zone had disappeared.

I had looked at the opportunities for conferences that I was going to take part in for 2020. 

Then "The Ludicrous Year" took over.

I had to adjust to doing conference talks on Zoom. I had to manage rooms of people I couldn't see, record conference talks to a camera with no one watching or listening, and then interact with people via chat to be able to see how well I did or if I did well at all. Frankly, this was challenging as I tend to work off of a room of people and their reactions to see what is needed and adjust my message, even if I have practiced it numerous times. I also got to work with video tools and presentation tools in a way that I hadn't done so before, so there was a lot of learning and new approaches I could look to.

My family reality had looked like it would be productive and fun. Two of my daughters were working at San Francisco International Airport, with carriers working out of countries they were looking forward to visiting and perhaps even working in for a bit. My son was working on setting up a studio with some other friends and investors in Los Angeles. Christina had several opportunities with the law office she works with and on the whole, 2020 looked like it might be an adventurous year.

Then "The Ludicrous Year" took over.

Both of my daughters were furloughed from the airport (makes sense when nobody is flying or flying in such a limited capacity, especially with International carriers). My elder daughter decided to take the opportunity to pursue a cosmetology school program. We set up the upstairs living room area to be her online classroom and her own salon area for learning and practice. My younger daughter found work at a couple of places outside of the airport and has a small recurring role in the airport concierge service that she can do a few hours each week but it's definitely not the career she had been working towards. My son had to hunker down and get creative for ways to work in an industry where performances and recordings and touring were the main bread and butter of the industry. He's said it's been a challenge, added to the fact that he was the one out of all of us thus far to actually contract COVID-19. One of his housemates brought it home to all of them and thankfully his case was mild, though he said he wouldn't want to see anyone go through the six weeks or so that he did dealing with it.

For me as a whole, this year felt like treading water. Much of what I enjoyed doing had to be bottled up and thus I tended to likewise treat myself that way. Still, I could say that there were several neat developments:

- The Testing show returned to twice a month publication, so I was able to keep busy with that.

- I had some opportunities towards the end of the year to write for some publications I hadn't worked with before.

- The fact that I wasn't performing or traveling to festivals meant that the money I would have spent over the course of the year didn't get spent and that allowed me to revamp my studio/rehearsal space and buy new equipment for a guitar rig that I had dreamed of owning for years, and the oddness of "The Ludicrous Year" made that both more possible and actually more affordable, since used gear shops were filling up dramatically this year.

- One interesting development, and for those curious, is that I joined TikTok and started making content about "Heavy Metal Vocal Technique", among other things. That has also caused me to revamp my home studio space (about thirty times) so as to make it more interesting for filming and presentation purposes. If you'd like to see me doing my thing there, it's @mkltesthead :).

Needless to say, this was not the year I had hoped for and I do not want to make light of the fact that people lost loved ones this year or literally died this year. I have seen both the best and worst of human behavior. I have celebrated highs and dealt with crashing lows. Still, I am here, all of my family is still with me, I'm gainfully employed while many others aren't, and I can look forward to 2021 that I am again hoping will prove better than this year now closing down. 

Be excellent to each other :)!!!

 

Get the Funkify Out: A Neat Accessibility Tool/Disability Simulator

 Are you all sick of me yet? Wow, that was a lot of writing/typing/conferring this week. to be honest, I've really missed it. I was happy to participate in PNSQC this year even in the unusual circumstances and challenging technical issues we went through. I will talk more about that in another post and also in the next The Testing Show podcast but for now, I want to share something a little new for me and maybe new for a lot of you all, too.

While I was developing my "Add Some Accessibility To Your Day" workshop, I reviewed the tools that I use regularly and looked to see if there was anything interesting out there I hadn't played with recently. Many of you know my general toolkit:

• WAVE Browser Plugin
• AXE Browser Plugin 
• Accessibility Developer Tools
• VoiceOver and NVDA Applications (MacOS and Windows 10)
• NCSU Color Contrast Analyser
• Hemingway Editor (yes, it is an Accessibility tool. FIGHT ME ;) )

I of course discussed these but I also found a newer tool that definitely interested me and that I've been having fun working with. That tool is called Funkify.

Funkify is a Chrome Extension and it is a little different than the tools mentioned above in that this doesn't really call out errors or find bugs... at least not in the traditional sense. This is a simulator that puts you in the driver's seat as any number of people with disabilities so you can experience your site through their eyes/ears/hands.

Funkify modifies your site so that you can see it as these personas see it. You also have the ability to create your own personas based on criteria that you deem important and you can adjust the level of challenge/disability.

For example, let's look at the W3C Before and After site.

How might this site look to someone with dyslexia? Funkify can give us an idea. Just press the toolbar and select the defined persona (in this case "Dyslexia Dani") and expand to see the options.

You can adjust the amount of jitter/scramble of the letters. Make it mild or severe.

Onvce you've dialed in the level you are interested in, let it run until you are satisfied (or dismayed, or annoyed, take your pick):

There are a variety of disabilities that can be simulated: color blindness, astigmatism, jittery hands, high distraction, macular degeneration, or you can create your own agent/persona with the criteria and level of disability that you are interested in simulating.

Give it a name and a description and save it. It's ready when you are.

This is not too far from my own visual situation. In fact, this looks very much like the page without my reading glasses.

In any event, if you want to have a simulator that will put you in the shoes of a variety of users, specifically those with disabilities, Funkify is worth a look. to be clear, the free version is limited, if you want to have all the options you will need to pay for the Premium version. still, if you want to have the opportunity to see what your site looks like in a variety of Accessibility scenarios, this might be just what yo uare looking for.

PNSQC 2020 Live Blog: Diversity in Testing and Technology: A Moderated Panel

 

Wow, three days go by fast when you are typing like mad. This is the last formal talk before my workshop and after my workshop, the conference will be over.

Tariq King is moderating this panel discussing the diversity in testing and technology and he's leading with "is it important if people aren't doing anything to promote it?"

Lisette has built several teams and she says that the teams need to match the users. If the users are diverse, then the teams need to likewise be diverse. Tariq also commented that diversity additionally means that designing for our users is often a way to drive revenues. 

Raj pointed out that the world of work has changed, with teams spread out all over geographical areas so diversity is of course important because diversity exists somewhat by default, even if it's not intentional. even with that reality, there is still a lot of work that needs to be done to more effectively represent our customers and to have their involvement in the engineering and testing process. 

Hussain In the individual contributor roles, there may be an overrepresentation of some groups but that definitely is not the case in management. Management skews heavily white and male. In ways, Education requirements can be a barrier, so google as an example is looking to offer a certificate program that, if implemented, will count the same as a degree. Rebecca mentioned that while this is a step, the question is how will that create new pathways up the career ladder.

Lisette points out that the individual contributor pipeline is relatively diverse but the people hiring are not diverse. When companies promote and help diverse candidates grow, they can become leaders in their organizations and become decision-makers for their companies. 

Raj points out that often, mandates are made to say that "x number of people need to be diverse" but there's little emphasis beyond the numbers. Hiring is great but what matters is the path and opportunities for the hires beyond just filling seats. where will these people go in the future? Will they be seen as valuable team members, encouraged to move up, and rise up in the organization?

I'm happy to hear about Accessibility and diversity including individuals with disabilities. Yes, my wheelhouse, so let me crow a little ;). there are specific and unique challenges when dealing with neuro-diversity and ability-diversity that makes this challenging. If there is an opportunity to interview a hearing-impaired engineer, how will that interview be conducted? Is the burden of interviewing on the interviewer or interviewee. If an interpreter is needed, who is on the hook for that?

Tariq notes that successful teams are gauged on the amount of time they allow each other to speak, as well as to allow for the gender of the team members. It's interesting that successful teams tend to have more women involved. This mirrors my original experience with Cisco when the majority of the testing team was done by women engineers. Was that a fluke of Cisco or was that a broader tend when I hired on? I'll honestly never know but it was definitely an influence on me and my work over the years. I've long said that if it weren't for a cadre of women who helped me over the years, I might not be in this industry at all.

Lisette say that to just assume a woman would be an excellent fit for QA is wrong. It's interesting to me because I had wondered over the years if I was the odd one because I wanted to be a tester over those years I was at Cisco or that I was actively encouraged by women over those years to participate because that was their world and they were welcoming me in. as Cisco grew, it certainly seemed that the overall population of testers that were women was not as pronounced, though I would still say it was significant by the time I left Cisco in 2001. 

As the father of two daughters and a son, I have actively encouraged my daughters to pursue the goals that they want to. One of my daughters wants to work in aviation, ultimately to become a pilot someday. To that end, I have encouraged my daughters every bit as much as my son and I actively want to see them succeed in whatever role they wish to participate in. 

I am curious to see if COVID-19 and the cratering of expectations of working in a specific location long term might have on the overall attitude of who works where and when. Will this help to encourage a more diverse group of people to apply for jobs, especially if the older ex[pectation of packing up everything and moving someplace is not as actively sought after. 

sadly, this is all i can track as I have a workshop to prepare for so I will bid adieu here but a great conversation and glad to hear it while I could :).

PNSQC 2020 Live Blog: Towards A Culturally Inclusive Software Quality with Jack McDowell & Ying Ki Kwong

It's been interesting to see how we have seen the world adapt over the past thirty years that I have been in the software testing industry. In some ways, it's a very different world and yet at the same time, not much has changed. When I came into the testing world, there was a need for understanding requirements, there was a need to know how to test and there was a need to execute a certain number of tests, manually if we must but automated would be spectacular.

What I just described was 1991. Does it really sound that much different than 2020? Let's take a look at some other factors. Who is doing the work? For me, I remember thinking about this as I was working for Cisco Systems and I was hearing how few women or PoC were actively involved in tech. I frequently wondered what they were talking about as I was literally surrounded by women as peers and women as managers, as well as a rather diverse team as related to cultural and ethnic background. It would take me several years and watch the company grow to have that lens be changed and see the point. Where am I going with this? It's sometimes hard to see the issues and challenges we face when we think that the world being described doesn't match our experiences. I made the mistake of thinking the early days of Cisco Systems were indicative of the world of tech as a whole. 

Again, interesting but Michael, what are you going on about? Well, it's key to understanding Jack and Ying Ki's talk (and Jack is the one delivering it but since Ying Ki helped write the paper, I'm giving him credit, too ;) ). In any event, the point is that we see the world through our lens, and that lens may be accurate or not but we won't know it is accurate if we don't have other points of reference. With this idea, we should also be asking "what does quality actually mean?" It means whatever the team at large decides it means, and then we radiate out from there. However, we are missing a bit of the point that quality is going to be skewed if we don't seek to pay attention to what others think that quality is.  

An idea that Jack presents is that we not look at trees as a metaphor for meaning but instead look at rhizomes. A rhizome is a plant that spreads out a root system and buds, vines, etc grow out at a variety of points. we might look at these as being multiple plants but in truth, it's the same plant. 

Ha! Communities of meaning using Accessibility. We're in my court now (LOL!). Well, not really but it is interesting to see what examples for accessibility are used and what actually counts. Jack is correct that accessibility often looks at disabilities as a monolith when in reality, there are a variety of unique disabilities that require very different methods of accommodation. I'm all too familiar with the approach of "load up a screen reader, walk the application and call it a day". Yes, I've been there and so have a lot of others. The fact is, that approach leaves a *LOT* of things out of the mix or even giving consideration. Sight issues are not the same as hearing issues, and they have little to do with mobility issues, and those have little to do with cognitive issues, and on and on.

So Standards are often included in these quality questions. Standards can often be helpful but standards also tend to take a prescribed way to identify what makes for software quality. yet the question remains, does a standard take into account cultural differences of other approaches? Jack is arguing no and I'd say I agree with him. Jack just said that Agile Software Development is pretty much Anglo-centric. Yang Ki refutes that the way that agile software development works in the USA would not be culturally acceptable in Honk Kong, for example. The idea that individual engineers would be able to express their opinions or desires about products and openly question management. It also shows that in other cultures the Agile principles can be subverted and exploited. 

Again, the question we want to consider is "how does culture affect quality?" The answer is there is really no monoculture in all of this, there is a mixture of cultures that blend together and often help define these communities of meaning. How this is actually accomplished may well be a long and involved process but it is interesting to consider other communities of meaning to address these issues.

PNSQC 2020 Live Blog: Are You Ready For AI To Take Over Your Automation Testing? with Lisette Zounon

 

All right! How is that for a title ;)? I give props for coming out swinging and yes, I am indeed curious as to whether or not AI will actually have a long-term impact on what I do as a tester or automation programmer. It feels weird to say that but since "Senior Automation Engineer" is my official title, yeah, I kind of care about this topic :).

 

Many tools are built around allowing us to automate certain steps but in general, automation excels in the areas of the rote and the everyday repeatable. Automation is less good at dynamic environments and where there's a lot of variabilities. However, perhaps a better way to think about it is that automation struggles with areas that *we* feel are dynamic and not rote. Machine Learning can actually help us look for the repeatable, perhaps more specifically in areas that we are not currently seeing those patterns.

 
We are seeing the growth of pattern recognition tools and visual validation. As we get further into the process, we see that there are more uses for visual validation tools. It's not just is the picture in the same place. My question would be how can we leverage these tools for more approaches? As in most situations, a lack of imagination is not necessarily a lack of adventure or spirit but more often a lack of relevant experience. We tend to get mired in the specific details and we tend to look at these little tedious issues as taking too much time, requiring too many steps, or not being flexible enough to actually be useful.

Lisette makes the case that AI can help with API tests. Since API tests can be bounded (as in there's a set of commands and those commands can take a set of parameters, etc. So they can be generated and they can be called. In addition, auto-healing tests are often touted as a step forward., I will confess I have seen self-healing tests in only a limited capacity but that has more to do with what we use and what we currently test with rather than what is available and usable. I want to see more of this going forward and interact with it in a meaningful way.

I hear often the idea that AI will help to create more reliable automated tests. This comes down to agent counts keeping track of what is right or wrong by the definition of the software. It sounds cool on the surface but again, I'd love to see it in action. Lisette makes a good point that automation for the sake of automation doesn't really buy us anything. Our automation needs to serve a purpose so putting these AI tools to help us find critical paths or critical workflow steps and processes is worth the time. Again, I'm willing to give it a go :). 

PNSQC 2020 Live Blog: Let’s Focus More On Quality Engineering And Less About Testing with Joel Montvelisky

 

Joel starts out with a very blunt question... what value do you provide to your company? Not in an abstract sense but in a cash sense. Does your being there benefit the company in a bottom-line manner? It's a tough question in many ways because software testing is a cost center. No matter how we want to look at it, we are a cost of doing business. Granted, that cost may very well prove to be justified, especially if we find something that could be disastrous if it were to get out but it's hard to define how much money software testing earns for the company. truth is, unless our company is in the business of selling testing services, we are not really earning money for the company by doing testing.


this means that many testers often do something other than "just test". that has been the case with me many times, in that I put on a variety of hats when needed. sometimes that hat is tech support, sometimes it's systems builder, sometimes it's build manager, accessibility advocate, fill in the blank. In short, we do our best to add value in a variety of places but unless we are directly connected to the health of the organization and the generating of revenue, we are one step removed from the process and our value is less obvious and sometimes not recognized at all.

 
Joel makes the case that changing models of software development are applying pressures to the traditional role of a software tester. In the process, many testers are morphing into Quality Engineers. That's cool and all but what does that involve? Ultimately, it's that we work with developers to test and deliver products quickly, get feedback from the field, and continue to help develop software quickly and with high quality. Notice Joel is not saying "stop testing" but focus on additional areas that go beyond traditional testing. 

As I was listening to this, I kept thinking "oooh, this sounds so much like the A/B Testing podcast. and sure enough, that's EXACTLY where Joel was going with this (LOL!). For those not familiar, Alan Page and Brent Jensen host the A/B Testing podcast and they are also champions of Modern Testing (MT). In many ways, the move to DevOps is helping to emphasize the need and focus of testers to move into a more MT paradigm. There is still testing but the testing is less tied to individual testers and testing teams. I've witnessed this in my own organization. We have a testing team but all of us are individually embedded into separate teams. Our focus is less that of a testing team that reports to a manager and is the center for quality. Rather, we are often one or two people embedded in a specific team and while we test, we often work with developers to help them focus on quality and quality aspects. 

As we focus on quality and less on testing, we will find that we may be doing less rote testing but a lot more quality initiatives. We can be there early in focusing on developing quality requirements, we can get involved early and help programmers with questions and considerations as they are writing the code, we can get involved with automation and infrastructure so that much of the rote and humdrum steps can be codified and taken out of our front line attention. We can be involved in the build management and CI/CD pipeline and make sure it is working cleanly. I appreciate Joel including advocating for Usability and Accessibility. He also emphasizes having testers be part of bringing customers into the process, whether that be directly or indirectly. the better we understand the business cases and real-world uses, the more effective we can be at addressing the real quality issues. We also have the ability to teach developers about testing and test principles. I have myself had a number of situations where I've shared testing ideas and the developers have said "oh, that's cool, I've never looked at it that way." Be aware, though, that some developers simply do not want to test. That's OK, but it also needs to be clear that we are moving into other avenues and that if they don't test, there's no guarantee that we will either.

One thing's for sure, the future is going to be fluid, and to borrow from Ferris Bueller, "Life moves pretty fast. If you don’t stop and look around once in a while you could miss it."

PNSQC 2020 Live Blog: Test Engineers are the Connectors of Our Teams with Jenny Bramble

 

If you know Jenny, you know her enthusiasm and her energy. there's no way I'm going to capture that here, but I will do my best to capture her message at leat. 

Testers are in a unique location to bring the squad together, so to speak. In my musician days back thirty years ago, one of my bandmates said that I was the "psychic glue" that held us together. In many ways, I feel testers can be that "psychic glue" for an organization. However, to be psychic glue, it has a specific requirement and that is that we must be engaged. If we are not engaged with our team and with the other people in our organizations, we lose effectiveness.

Testers can be at the heart of communication but again, we need to make sure we are communicating to be actively involved. We talk about requirements, discuss bugs, present new features, and give the user’s perspective. No one else really has that level of reach, so let's make sure we are aware of that place we hold, and LET'S USE IT!!!

Teams are built on communication and teams fall apart without it. Sometimes I have found myself in situations where I have had to push into conversations. The fact is, we are welcome but we are not always invited. I think at times that's because we are often the bearer of bad news, or to borrow an old quote, no one likes the person who calls their baby ugly ;). that means that the ability to openly and honestly communicate belongs to us and it means we need to be careful but also deliberate in our interactions.

 

I have had experiences with being a musician, with being a competitive snowboarder, with being a scout leader, with being a husband, and with being a parent. That is on top of being a software tester. those are all contexts that require communicating in a little different. the way I communicate musical ideas is not going to work the same way as I communicate to scouts. Likewise, the other way around. It's not that the method of communication is all that different but it's the familiarity with concepts and approaches. Sometimes, I can just say a phrase, and my bandmates will know immediately what I mean. The reason for that is that we have interacted a great deal so we have stubbed our mutual toes a whole bunch. we've had lots of false starts. Because of that, we have a high tolerance for each other's eccentricities and that helps us communicate quickly. However, I know for a fact I could not communicate that way to the scouts I lead, or with the developers on my team. Familiarity breeds contempt, sure, but there's a benefit to that so-called contempt. I rarely get to that point with my scouts or my developers. thus I need to be a little more reserved in those capacities. On the other hand, I find that I can be more focused and granular with scouts and developers the way that my bandmates would pick me up and throw me out the door. again, style is important but communication is essential.

Jenny is highlighting that some people want a lot of details, while others want a high-level summary. A neat example she shared about how to work with a summary person is to have them review emails you plan to send. Sounds odd and awkward but at the same time, I get how that would work to get their attention and to give you hints as to how they prefer to communicate. Jenny also explains that she is a story based communicator (I love this revelation because it reminds me so much of me). The way she is describing this reminds me a little bit of "it's a baby penguin, caught on an iceberg"... and if you are not on TikTok, that may not make a lot of sense but yeah ;). the point is, invest in people and they will invest in you.

Regardless of how good we try to communicate, we often miss the mark ("What's a penguin? What's an iceberg?!"). It's not because we are dumb or indifferent, it's because we may jost not have hit on their preferred way to communicate. Some people don't like speaking but do great with texts. Some people struggle with email but do great on the phone. You just have to keep trying and find the communication mechanism that works. In short, if you intend to be a good dose of psychic glue, be prepared to communicate and be prepared to put in the time to learn other communication styles. By doing that, you can be that indispensable member of the team that no one could see functioning without you.

PNSQC 2020 Live Blog: Quality Engineering – Reflecting on the Past & Present to Accelerate the Future with Dr. Tafline Ramos

Ok, I hereby confess that the last talk was a hard one to wrap my head around so I'm looking forward to getting my head back on straight. Here's hoping this talk helps to do that (btw, just kidding about the last talk but I confess I've never considered floating point in depth in such a manner ever).

Our closing keynote tonight is the evolution of modern Quality Engineering. To understand where software quality is heading in the future, it helps to understand what we thought of Quality Engineering in the past.
 

It certainly seems that the conversation of Quality is circular. Many of the ideas of people like Deming and his points through to Margaret Hamilton and Glenford Myers. Somehow, the idea of testing being removed from development as a standalone discipline, and now we seem to be working to merge testing back into more traditional quality standards. the problem with having a dedicated tester (and so many of them for specific issues) that we created and refined the role of the tester to be at the tail end of the quality process. This reminds me of the space of testers in the early 1990s and how many there were vs the latter part of the 1990s and how many software testers there were and that specific separate group. 

Granted, I can only talk about the Cisco Systems approach in the 1990s as it was the only tech company I worked for at that point in time. Early in the days of Cisco, the software testers were not a dedicated group. Everyone who was a tester did something else in addition (my early testing years also included me being a Lab Administrator). I didn't get designated a Test Engineer proper until 1994 and that was when many others became branded as testers in a software sense and that being the core purpose for our work.

I've also seen that there does seem to be a divorce between the idea of building in quality upfront, and that there seemed to be a reliance on testers being the main line of defense for bugs. I remember that attitude and approach well, and in many ways, it is still that way. Ideally, we need to create Quality as a process at all levels. we need quality all the way at the very beginning of the story workshop that proposes a new feature. It needs to carry over through development, through having everyone at every level associated with quality and making it a core part of what they do. 

To be clear, I'm not saying people go out of their way to not deliver a quality product, just that our practices tend to make that approach harder than it really needs to be. It's interesting how shifting the word Test to Quality changes the whole conversation.

Cool, so what is a Quality Engineer, how do we become one? we would need a range of skills that are functional, technical, and consulting related, to be able to work effectively with a  number of groups, not just our own and not just related to testing. they need to be effective in methods, techniques, tools, and approaches that allow them to be effective in a variety of areas. A Quality Engineer also needs to beef up skills in technical areas, specifically coding. 

The fact is a large percentage of projects fail, badly enough to threaten a company's existence. A poor scope on requirements and quality overall ranks high in the reasons for this. There are a lot of techniques that can be used to help mitigate these failures. To reduce the overall cost of quality, it would be much more helpful to focus our energies early in the process, and let's stop the idea of testing at the late stage of development. Move the testing further back in the process and make quality the most important discipline in all groups as far back as possible.

 

PNSQC 2020 Live Blog: Testing Floating-Point Applications with Alan Jorgensen

OK, now I'm getting mildly anxious (LOL!). Hearing about floating-point errors being disastrous always makes me nervous because I frequently look at calculations and think to myself self "how precise do we need to actually be?"Also, wanted to give credit to this talk being presented by Connie Masters.

I've always operated on the goal that anything beyond tens of thousands is not that critical (heck as a little kid I learned Pi as 3.1416 because anything more precise was just not relevant (thanks, Grandpa ;) ). However, I know of course that in certain applications (chemistry and astronomy) there are of course considerably more points of accuracy with significant digits.  having to rethink this now, as I feel with digital systems and with huge sample sizes, rounding errors that are insignificant on their own can stack up and be a real issue. Also, at what point do we go from insignificant to catastrophic?

This is the first time I've heard the phrase "logarithms are the way to ad apples and oranges" and I'm not 100% sure I get the implication but it's a definitely memorable phrase. the only thing I am feeling for sure is the fact that I feel like what I've understood as discrete mathematics all these years is lacking... also, I don't quite know how I feel about that.                              The net result of all of this is that I need to get comfortable with bounded floating-point and get some practice with it.

PNSQC 2020 Live Blog: Quality Focused Software Testing In Critical Infrastructure with Zoë Oens

We are all familiar with the "Iron Triangle" where we get three sides; Faster, Cheaper, Better... Pick Two, because that's all you will be able to achieve.  One hundred percent coverage is unachievable. Especially if the goal is to save money in the process. still, what is the answer when you have to test software on a  critical system? when I say critical, and when Zoë says critical, we are talking things like the power systems, water, medical, banking.  Bugs in production are not trivial.

So what do we do when it comes to testing CI (critical infrastructure) software? How close to one hundred percent coverage can be achieved? If a feature fails, what is the fallout? How many are affected? While not specifically software related, some of you may know/remember that I have direct experience with a Critical Infrastructure failure in my community. My town of San Bruno suffered a major gas line explosion in 2009. Maly lives lost and many homes destroyed. It took years to rebuild and in many ways, the scars and memories are still fresh. 

When we are looking at testing in these critical areas, we have to be able to prioritize and determine the mission-critical stuff. Granted, we can't just declare everything as mission-critical but dealing with the electricity grip or gas supply, yes, critical becomes more meaningful. 

Zoë mentioned her time in manufacturing helped her approach the questions and issues where CI comes into play. Make sure that there is a spread of knowledge. CI is an area where there should be no silos. No one person should be the one to know how to fix problems when they occur, both at a coding and an operational level.

An emphasis on test writing, test setup, and test execution needs to be codified and well understood, as well as run frequently and aggressively to be sure that as many cases have been focused on and addressed as possible. Automation falls into this spare especially if the steps are codified and well known. anything repetitive, especially if it is rote repetitive, should be automated. The goal in CI environments is to be able to run testing consistently and effectively. While there is an upfront cost here, this cost can be amortized over time, especially if any of the tests are long-lived.

CI environments can sound scary but the key takeaway to me is to be mindful and diligent, as well as look for repetitive areas that can be codified, confirmed, and repeated.

PNSQC 2020 Live Blog: Teaching Testing To Programmers. What Sticks, And What Slides Off? with Rob Sabourin and Mónica Wodzislawski

I had a feeling that as soon as I saw who one of the presenters was for this talk that "OK, this is going to be fun!"

I've worked with Rob over the years when I was part of the Board of Directors of the Association for Software Testing and I remember talking with him about this very topic. There's no question that testers can learn how to program and work as developers. It also stands to reason that developers can learn the skills to become testers. 

Mónica Wodzislawski is new to me but I have learned recently about a vibrant community of software testing centered in Montevideo, Uruguay, and I can only assume Monica is actively engaged in that community.

Corporate initiatives to “shifting left” and the focus on Agile software development, Test-Driven Development, and other areas pay lip service to softeware testing but do not approach testing in the manner that a skilled tester would. That does not at all mean that good developers cannot be good testers. They most certainly can and many are. The idea of actually applying software testing as a discipline within the Computer Science curriculum is a good one. Over the years, I've heard only a few places where testing is even mentioned beyond writing unit tests. The idea of a semester-long course dedicated to software testing (akin to the BBST course of classes) is still a foreign concept in developer's learning journies. 

Some of the challenges that get in the way of testing are the notion that somehow software testing can't be seen as universal (and to be fair, it cannot, there is no one size fits all approach). It seems that there is a hangup on the tool stack. If you don't understand all of the tools then you can't do effective testing. There are going to be unique tool needs and there will be unique challenges related to specific tasks but even with that, there are many testing methodologies that can be leveraged regardless of the software stack being used. 

Often the approach to templating and patterns is heavily emphasized with some developers. Others tend to look at the code and automation as the end result. Automation is a good tool to have and it's important to understand and have them but there is a lot of software testing that doesn't involve automated testing.

Rob and Monical both emphasize looking at risk to help determine areas that might need testing at a preliminary or expanded level. Create models that stand up to the work that you need to do. By developing a model, we encourage developers and testers to focus their attention on the areas that are going to be the most vital to the business. There's plenty more to do but that's a good place to emphasize. 

By setting quality goals and examining case studies, software developers can learn a number of great skills including building conceptual models, examining and applying patterns and anti-patterns, and examining common faults and solutions that can be applied.

PNSQC 2020 Live Blog: Testing COVID-19 Models: Getting Important Work Done In A Hurry with Christian Wiswell

Now, this is interesting and timely :). I had wondered if there would be sessions specific to the biggest challenge we have faced in decades as far as health and public safety is concerned. Looks like I was not to be disappointed.

 



COVID-19 has disrupted a large number of things for a lot of people and the odds are high it will continue to do so for a long time. However, how are we able to make these determinations? How do we respond to these situations? How do we determine where to intervene? To do this, we need to use and refine computational models. I confess I have always wondered how these models are created and more to the point, how would we test these computational models?

To be frank, the idea of testing this is hard for me to get my head around. How do we determine if the model created will map to reality? It seems to me that being able to track data and getting actual results seems to be a constant chase for data. Christian points to the fact that getting a point for point confirmation is impossible. So how does this work?

The key to this is that going from exposure to infected state, to treatment time, to recovered state, is logged and is tracked. based on these values, statistical tests are created and run. These tests are run multiple times and they are tracked to see how they behave. In short, there is no genuinely "true" test, but there is a way to confirm that the results are predictable and consistent. It takes a lot of time and testing. Over time, we get used to how these tests run and we get a general example for how a disease behaves. COVID-19, however, acted in ways that these models were not totally prepared for. While we had warnings in November 2019, the modeling scenarios for other diseases were inadequate for the novel Coronavirus. It behaved differently enough where the software models were not up to the task, as well as finding the team in lockdown because of the spread of the disease (wow, that's a pressure I've never had to deal with).

Covasim is what it sounds like, it was a Covid simulator with custom code that Christian had not worked with before. there were unknown dependencies and properties for the code was going to take time to learn about. To start, there were a variety of parameters that could extend to support classes. One challenge was to be able to change the vocabulary so that code tokens were more easily spottable (such as mapping the variable n to "NumberofInitialParticipants"). the first test was a way to get data about the random seed). The test to confirm the random number being pulled before the random number was populated helped to determine if the tests could be repeatable. One of the most important things to look at was the "test exposure to infectiousness delay deviation scaling". In short, how long did it take from the time of exposure to the time of being infections, whether or not symptoms were showing? Over time, with more data and more tests, the confirmation that the general mean of the experiments and tracking data became more consistent. In general, it meant that more people would be able to be predicted as to when they would become infected and be infectious. 

One of the key things to realize is that you are never testing if the software is correct or behaves correctly, but whether or not the configuration is predictable and the model matches real-world behavior over time. It's also important to configure unrealistic scenarios so as to see if the outputs are either in line with the values provided of that the outliers are so out that the exceptions actually prove the rule.

I confess freely this level of testing is outside of my wheelhouse but I'm greatly appreciative for this talk. I've always wondered how these tests were determined. I now feel I have a better understanding of that, if only a little ;).      

PNSQC 2020 Live Blog: Generator-Based Testing: A State By State Approach with Chris Struble

Over the past several decades, we have seen a shift away from manual testing to automated test execution. While automation can help us take out a fair amount of the tedium of everyday testing, what we still deal with is test design at a manual level. Most of our tests have to be considered and have to be designed to be effective. Examples are good and testers are god at this but again, we are not terribly patient so we don't necessarily do a broad number of tests because, well, we are lazy.

  

What if, instead, we could generate test cases based on software coming up with more examples than we can provide? Testers create a description based on the tests they want to run. based on the data the tester provides, systems would then be able to generate additional test cases and generate multiple examples of tests we can run. I do some stuff like this currently and manage a number of for loops with a list of areas in want to focus on. that's a rudimentary level of this, but what Chris is describing goes considerably farther.

Our typical test case matrix focuses on Example-based testing (EBT). Test cases are manually designed one example at a time. we can then fill in parameters for those tests and then execute the tests we create. the problem is, of course, EBT can only test examples that are part of my matrix. It doesn't test for things I did not include.

Generator-based Testing (GBT) works by providing our descriptive tests to a system and that system can then take what we provide and create random tests that muck with the data or creates oddball examples (fuzz testing fits in this mold). 

I've had a dream since reading James Whittaker's book "How to Break Software" many years back. He has a first maxim that I have always loved, which is "look through the code and find every error message listed. Figure out what is meant to cause that error message to display, and then get that message to display through software interactions at least once." I'm paraphrasing but you get the idea. How cool would it be to be able to have a tool that would specifically look for error messages, determine where in the code those errors were supposed to display, and then give me a matrix of options related to automated tests that found be run to surface that error.

This is an interesting thought experiment and I confess not one I have given a lot of attention to. 

PNSQC 2020 Live Blog: Breaking Down Biases and Building Inclusive AI with Raj Subrameyer

 

All right, here we go with Day 2!

First of all, I want to give props to Joe Colantonio and everyone else for the management and efforts of yesterday to keep everything on track. For those who are wondering what it is like to do a conference totally online, it's not always seamless but Joe handled issues and problems like a pro.  Some things that have been interesting changes:

There is a Virtual Expo so if you want to see what vendors are showing and do so with your own time and focus, you can check out the Virtual expo by clicking here.

The question and answer is being handled through an app called Slido and that makes for a clean way to ask questions and interact with each speaker rather than have to try to manage a Zoom chat feed. Again, a neat approach and well presented.

So for today's opening keynote, it's exciting to see friends that I interact with getting to be keynote speakers. Raj Subrameyer and I have interacted together for several years. He was also a recent guest on the Testing Show podcast talking about Tech Burnout (to be clear, not the topic of today's talk). If you'd like to hear our interview with Raj, you can check that out by clicking this link here.

Raj's talk is focused on building Inclusive AI. Sounds scary, huh? Well, it doesn't need to be. He opens with using three movies (2001: a Space Odyssey, HER, and Ex Machina). What was interesting about these movies is that they were science fiction and now, they are science fact. the point is sci-fi has caught up with our present. The question we might want to ask is, is this a good thing? It all comes down to how you look at it. Are you using Siri or Alexa regularly? To be honest, I do not use these very often but I have worked with them, so I'm not a Luddite. Still, there is a small part of me that doesn't want to rely on these tools just yet. Is that a fear-based thing? A trust-based thing?  Maybe a little bit of both. Do I really want to have these AI systems listening in on me? Well, if I'm someone who uses apps like Google, Amazon, Facebook, Instagram, or TikTok (hey, don't judge) I'm already training these systems. Alexa is just a voice attached to a similar system.

let's face it, technology can be creepy. It can also be very interesting if we understand what is happening. AI systems are getting trained all of the time. facial recognition, text recognition, voice recognition, these all are tweaked in similar ways. As Tariq King explained in a talk last year at Testbash in San Francisco, it's not anything sinister or even terribly complex. Ultimately, it all comes down to agents that keep score. When an agent gets something right, they keep a counter of the number of times they have successfully guessed or provided the right answer. They likewise decrement counters when they get things wrong. Over time, the counter helps figure out what is right more times than not. It's not perfect, it's not even intuitive, but it's not really super-human or even all that complicated. we just tend to make it and treat it as such. 

Raj points out that the neural network inside of each of our brains has a number of synaptic connections that, when calculated, equals the number of stars in our galaxy (maybe more) and to quote James Burke, "everybody has one!" The most powerful computers still pale in comparison to the connectivity and plasticity of a single human brain (though interconnected systems can certainly match or exceed single brains).

AI can be classified as weak and strong. Most of the systems that we interact with currently are classified as Weak AI systems. Yes, they can be trained to give a response and they can perform specific steps. Systems like Deep Thought can play chess and beat the best human players, but that is still an example of Weak AI. In short, the system can brute force avenues and do it fast, but it can't really "think". Strong AI can think, and emote, and sympathize, and deal with situations in dynamic ways the way people do. So far, there are very few AI systems that can do that, if any, really.

I'll use an example from my own musical life. I've recently been shopping for guitar amplifier heads, older ones. My all-time favorite guitar tone ever comes from the Marshall JMP amplifier head, which was popular in the early to mid-1970s. Additionally, I also very much like the Carvin X100B Series III amplifier head. A Weak AI would be able to compare specs of both amps and give me a readout of which amp may have the best reaction to fault tolerance or to sonic frequencies. It will not, however, be able to tell me which amplifier head "sounds better". That's a human judgment and it's not something that data will necessarily be able to provide an answer for.

We may be familiar with the study that was done where resumes were submitted using both typically "white" names and also resumes with "black names" (or traditionally seen as white or black names), the AI system was trained on the group of data and, interestingly, it would reject resumes with "black" names twice as often as it would "white" names. That definitely invites a question... how did the system "learn" to do that? Was it trained to do that purely based on the text in the resumes, or did some bias enter the system from the programmers? It's an interesting question and hey, I know what I think about this (hint: humans biased the system but I asked a Slido question, so let's see if it gets answered later ;) ).

Another thing to consider is that AI can be abused and it can also be fooled. In the world today with applications like Photoshop and video editing, deep fakes can be created. Provide enough deep fakes and systems can be trained with literally fake information and those systems can develop agent counts that are not based on reality. Scary but definitely feasible.

Ultimately, AI is as good as the data it is provided and the people that program the systems and the algorithms that train them. Systems can "learn" but again, learning is having a weighted count of something. the more it's "right", the higher the count, and the greater the odds that the "right" answer will actually be "right" in this case. Interesting stuff, to be sure but I'd argue that the odds of these systems coming together and replacing human intuition and interaction is quite a way away. that's not an invitation to be complacent, it's a recommendation to spend time to learn about these systems and how to better understand them and interact with them, and also that we have a responsibility to make sure that the systems we build are not just good quality but also fair to everyone.

PNSQC 2020 Live Blog: “Rethinking Test Automation” with Paul Gerrard

I just realized that the last time I saw Paul in person was back in 2014 at Eurostar in Dublin, Ireland. I was really looking forward to seeing him again after so long but alas, I guess this will have to do.

It's interesting to see that the notion of being surprised about the troubles related to testing automation has been with us since the nineties at least (and some could argue even longer as I remember having issues and dealing with oddities back in the early 90s as I was first learning about Tcl/Tk and Expect. We still struggle with defining what test automation can do for us. Sure, it can automate our tests but what does that really mean?

Tools are certainly evolving and look nothing like the tools we were using 30 years ago. Still, we are dealing with many of the same principles. The scientific method has not changed. I share Paul's criticism that we are still debating what test automation does and what testers do. the issue isn't whether or not our tests work, it's whether or not the tests we perform are actually gathering data that can be agreed to or refuted for hypotheses. As a tester, we want to either confirm or refute the hypothesis. At the end of the day, that is what every relevant test needs to do. We can gather data but can that data that we gather give us meaningful information to actually tell us if the software is working as expected? One could argue that assertions being true are passes... but are they? They are proving we are seeing something that we expect to see but is it actually proving a hypothesis, or merely a small part of it? In short, we need people to look over the tests and the output to see if it is really doing what it should be.   

Paul suggests that we need to move away from scripted tests to more modeled based tests. OK, but what does that actually mean and how do we actually do that? Paul makes the assertion that tools don't think, they support our thinking. What if we removed all of the logistics around testing? If stripped of our usual talismans, what would we do to actually test? rather than stumble through my verbiage, I'm stealing Paul's slide and posting it here:

The key here is that test automation misleads us, in that we think that tools are actually testing and they are not. What they are doing is mapping out the steps we walk through and capturing/applying the sets of data and results that we get based on the data and actions we provide. The left is the exploration, the right is the evaluation, the middle is the testing or the setting up so that we can test. Automation won't work if we don't have a clear understanding of what the system should do. Paul is emphasizing that the problem and the area we need to improve is not the execution of tests (our tools can do that quite adequately) but in test design and test planning. In short, we need better and more robust models. 

The old notion of the human brain is that it is brilliant random, unpredictable but slow and lazy. Machines are literal, unimaginative, but blindingly fast and able to do the same things over and over again. Combined, we have a formidable combination. 

So what do we want to see the future be for our tools? First of all, regression testing needs to look at impact-analysis. How can we determine what our proposed changes might do? How can we stop being overly reliant on testing as an anti-regression measure? How can we meaningfully prove functionality? Also, how do we determine the optimal set of tests without guessing?

Paul makes the case we need to understand the history of failures in our tests. Where can we identify patterns of changes? What are the best paths and data to help us locate failure-prone features? Manual testing will not be able to do this. Machine learning and AI will certainly get us closer to this goal.

In short, we need to move away from passive to active collaboration. We need to stop being the people at the end and work towards active collaboration. We need to be able and willing to provoke the requirements. we also need to create better mental models so that we can better understand how to guide our efforts.

PNSQC 2020 Live Blog: Full Stack Testing Is A Culture with Christina Thalayasingam

Yay! another new speaker. Christina has said that this is her first time speaking at PNSQC. To which I say, welcome, and thank you for joining us.

Full-stack is a term that I have heard used mostly with developers and over the years I have joked about the development of "full-stack testers" (I recall Mark Tomlinson having a spirited debate on that term some years back ;) ). Still, that is the core point of Christina's talk so what does it mean?



My first question is "do we want this to be in one person, or are we talking about the full stack being in the team as a whole?" Is it realistic to have a full stack tester? Can any tester know everything from top to bottom? Or are we looking at full-stack testing meaning any tester or the team of testers can attack any given testing challenge? 

As Christina points out in her abstract, there are several areas and types when it comes to Quality Assurance

Testing
Test Automation
Performance
CI/CD/CT

Usability
Accessibility

to be frank this is an interesting and exciting prospect. It reminds me that I should probably review Alaister Scott's "Pride and Paradev" as that was one of his goals, to help encourage testers to branch out and become broader jacks of all trades. Basically, it comes down to any tester being able to tackle any testing challenge anywhere up and down the development stack. It is unlikely one tester is going to be able to completely cover everything out of the gate. However, it's a good bet if you have a team of three to six testers, you may well have one full-stack tester combined. From there, make a commitment to developing the team to spread that knowledge so that everyone can be competitive. Of course, the odds of getting all six team members to be equal across the board for all skills is probably not reasonable but it's a great goal to reach for. My guess is, even if you don't completely make it, there's a great chance the team will be far more formidable than they were previously :).

PNSQC 2020 Live Blog: Communication Is Key: Lessons Learned From Testing In Healthcare Technology with Rachael Lovallo

As we consider communication challenges, what would we do if those challenges didn't just end up being an issue of finding or missing bugs but potentially being an issue that could spell the difference between life or death? Did that get your attention? Yeah, it got mine, too.

Rachael Lovallo focuses her quality initiatives on the emergency healthcare industry. As she clearly stated at the beginning "we can’t afford bugs, as they could endanger real humans". 

Make bugs as bullet-proof and reproducible as possible. 

This may seem obvious but I would guess that missing anything or being vague would be something that would cause anxiety in this particular environment. By making sure that we identify all of the aspects that point to the bug, a reliable reproduction strategy, and an expected behavior or output to compare with is critical.

Talk to people writing the code you test

Again, seems evident but how many of us really communicate about the code that we are testing? How much of it do we really know inside and out?  How confident would we be to be able to go through and read the code and know exactly what it is doing? I know for myself that I wouldn't be able to do that all on my own,

Ask for a second set of eyes on your work

Oh, I can so relate to this! As someone who basically deals with data transformations at the moment and the methods involved in that covers so many tools and points of contact that it just isn't possible for me to know where everything is and what I should be looking at. Fortunately, the development team I work with are great about showing me the areas I need to hit and to also recognize where I find areas of friction or that are less than optimal for being able to complete tasks. By getting them to walk the workflows and steps with me I am able to uncover with them ways to make the overall flow work better.

PNSQC 2020 Live Blog: Of Machines and Men with Iryna Suprun

As is often the case at PNSQC, several of the talks are from people I have not seen speak before. Iryna Surpum is focusing her talk on areas of AI and Machine Learning. As we start her talk, we look at the fact that there are few tools available where AI and Machine Learning are prominent and prevalent for individual users. Some hallmarks of AI-based tools and what is being marketed are Codeless script generation, the ability to self heal with changes to the environment, meaning the script can collect data about elements of the application itself, and the ability of Natural Language Processing to be able to convert the documentation to actual tests (this is a new one to me, so hey, I'm intrigued). 

Comparison of Visual Output and the expected design is becoming more sophisticated. More tools are supporting these features and additional levels of comparison are being applied (not just pixel to pixel comparison these days). 

So while we have these changes coming (or already here, how can we leverage these tools or learn how to use them in the first place?

Example tools to try out for these comparisons were, Testim, Mabl, and TestCraft. What did they provide? All three allowed for a Quick Start so that they could learn and be able to automate the same basic initial test case. All of the tools had recording implemented, which allows for initial set cases to be created (testCraft had a few extra steps to be set up and utilized so not quite as easily started as the other two). Modifying and inserting/deleting steps was relatively fast.

So what challenges were discovered/associated with these tools? as could be expected, the Codeless Script Generation (Recording) is good to get started but its usefulness diminishes the more complex the test cases become. This is to be expected, IMO, as this has been the same issue with most automation tools that promise an easy entry. It's a place to start but getting further will require proficiency and experience beyond what the recorder can provide. Self-healing is a useful feature but we are still at a point where we have to be somewhat explicit as to what is actually being healed. thus calling it self-healing may still be a misnomer, though that is the goal. So how about self-generated tests? what data is actually being used to create these self-generated cases? This didn't seem to be very self-evident (again, this is me listening, so I may be misinterpreting this). An example is to check to see that links work and are pointing to literally legitimate end links. that tests to see that a link exists and can be followed but that doesn't automatically mean that the link is useful to the workflow or that it will validate that the link is relevant. People still need to make sure that the links go somewhere that makes sense. 

So even though we keep hearing that AI And Machine learning are on the horizon and are even here changing the landscape, there's still a lot of underlying knowledge needed to make these tools work effectively. There's definitely a lot of promise here and there's an interesting future to look forward to but we do not have anything close toa magic want to wave yet. In other words, the idea that AI is going to replace human testers might be a possibility at some point but that promise/scare is not quite ready for prime time yet. Don't be complacent, take the time to learn about how these tools can help us and how we can then leverage out brains for more interesting testing work.

 

PNSQC 2020 Live Blog: Capacity To Execute with John Cvetko

The first talk of the afternoon is being given by John Cvetko. The main point of John's talk is to look at the desire of an organization to change from a Waterfall development methodology to an Agile development methodology. for many of us, it may seem slightly odd that there are still Waterfall SDLC's out there but rest assured there certainly are, lots more than we might like to think. John focused on the considerations needed to make the changes necessary to make the move from Waterfall to agile.

The example organization had multiple release trains (36 release trains in total) and part of the initial process was to create a hybrid release train. They didn't go all in or replace every single release train. Rather, they took on the Agile transformation within a few release trains. They took an isolated and low risk approach so that no more than 20% of the projects would be affected. the Agile transition even with this focus and need was still going to be a challenge and that the implementation of Agile was going to require an entirely different way of thinking.   




This gets us to the title of the talk. The “capacity to execute” is the most important aspect, in short,"Plan your work and work your plan". More to the point, there's more need for planning to make a transition effect. Whether the approach is to make a huge sea change, or smaller waves or bubble up changes, there has to be the ability to plan and execute effectively. 

There are a variety of metrics that are looked at (an example is ISO 25010 Software Quality Standard) and which quality characteristics were chosen to determine the level of focus of the transition. 

By focusing on Scope, Schedule, Budget, and Quality, we as an organization can make a real determination as to what the transition will be able to focus on and to what extent. Just because we want to make the change doesn't mean we will be able to automatically get there just because we want to. Often the business context, the company culture, and the revenue and cash on hand will help influence those paths. 

There is often a Culture Clash that may occur when each team works to try to make these transitions. thus the intentions and plans to execute are doubly important at this time. 

Make no mistake, these are tricky and challenging transitions. It may very well prove to be worth it but it just as likely will not be easy.

 

PNSQC 2020 Live Blog: Software Performance And Load Testing Utilizing JMeter with Anna Sharpe

 

this is number two of the fast-paced talks this hour. Performance testing is often seen as a black art and an area that requires specialized skills. To be sure, doing performance testing at a high level on a regular basis will require special skills and often use tools that are specific and need to be set up in a way that everyone knows about.

However, that may be overkill for what an immediate team needs as well as to run smaller levels of stress tests that are not as all-encompassing. Another challenge is that, if you don't have performance testing skills, you may find it a challenge to getting those skills or getting people to give you the time to train on those skills. The good news is that you can start doing performance testing with a small amount of data and practice getting a handle on the skills that you want to learn. JMeter is a great place to start with this. It's free, it has a small footprint (relatively speaking) and it is friendly to running on smaller, more isolated environments.

JMeter is a well used and well-known tool that has a broad user community and active developer support. JMeter can also work with tools like Fiddler and Charles Proxy to help construct tests and control the responses. 

If at all possible, run your performance and load tests during off-peak hours if possible and especially to make sure not to run these tools on the actual production environment. If you have to use the production environment, do so at a time that coincides with a maintenance window or otherwise runs at a time when customers are not actively using the system. If you have a confirmed level of response that the server is meant to provide (say, 30 active users) a good rule of thumb is to try to run a load test based on the expected load of the system, at least at first. If the system doesn't fall over or requests don't fail, you may want to bump p the number of concurrent users./threads and see how performance changes.

This is a good reminder that testing with JMeter need not be a big involved process and can actually be fun.