Retro Book Review: Rethinking Expertise

When  I was in Sweden, back in November 2013, I sat down to a talk with James Bach and we got into a discussion about how to categorize ideas and thoughts as to what informs our testing and what key features we’ve brought to our respective games. He said he noticed that I’d read a great deal, and that he was curious as to what I was reading and why. As I was explaining the books I was reading, and why I found them valuable, he kept coming around to the question “that’s great, but how have you applied this to your testing?”, and I realized that, frankly, I was struggling with answering that. For the life of me, I couldn’t explain why I was having trouble. I knew they were helpful, and I knew that I was taking ideas left and right and applying them, so why was I struggling with this so much?

James pointed out that one of my challenges was that I was missing some key ideas as to how to position my own expertise and what I actually knew, versus what I thought that I knew. He also counseled me that, in some ways, because of working with certain domains, and doing so for some time, I may be equating experiential expertise with contributory expertise, and that the two, though we might want to believe are equivalent, really aren’t. With that, James handed me “Rethinking Expertise” by Harry Collins and Robert Evans and said “I think you might find this book helpful”. 

For those with a short attention span, I’ll save you a bunch of time… yes, indeed, it was! If you want to have an opportunity to take a deep dive into the idea, ideals and avenues of expertise, what makes an expert an “expert”, and how fluid and fraught with controversy that term actually is, this book is a must read.

For those with a greater tolerance for my wordy reviews, let’s start with the centerpiece of the book, which is the Periodic Table of Expertises. This is going to be a short description of the table, not a full breakdown of every element. The explanation of the table and the elements in it take up over 50% of the total book.

We start at the highest level with Ubiquitous Expertise. This is the stuff each of us does automatically. When we walk, listen, speak a language, ride a bicycle, swim, or perform a physical task like reading, writing, or using a computer,  if we take the time to try to explain what we are doing, exactly what we are doing, we may well find that words fail us. Why? Because these are skills we take for granted, i.e. they are ubiquitous expertises. They are part of our “bare metal” programming, of sorts. 

As we step further away from the ubiquitous aspects of expertise, we get into the areas where we have some control over how we describe them, because they are actively learned and nurtured in a way that we understand active learning and nurturing. Facts, figures, trivia, arcana, minutiae, story plots, and other gained “chops” in a given area all fall into this sphere. These experiences can be split up into two areas (Ubiquitous Tacit Knowledge and Specialist Tacit Knowledge). It’s in the tacit knowledge areas that most of us “think we know what we know”, but may still have trouble verbalizing the depth or significance of what we know. This exists on a continuum: 

• simple fact acquisition (think of remembering a fact from a Trivial Pursuit game you participated in, and knowing that fact simply because you played that game)
• popular knowledge (you heard about it on the news a bunch of times, so you feel like you know the topic well)
• primary source material (you bought and read a book on a topic)
• interactive expertise (in my world, I am a software tester, so I have a fairly good grasp of the parlance of software programming, and can talk a mean game with other programmers, but I hardly consider myself a “programmer” in a professional sense, though I could “play one on TV”)
• contributory knowledge (I certainly can talk with authority about the programming I have personally done, as well as several challenges and pitfalls I have personally experienced. My experiences with software testing also falls squarely in this area). 

At the higher levels we get to even more specialized expertise, some of which are directly in line with contributory expertise, and some that are interactive expertise, at best. These are called the meta-expertises and meta-criteria, and we can include the pundit, the art, movie & restaurant critic, and in some cases, the general public.

While all of these classifications are interesting, they show how expertise is fluid, and, also, that it can be faked. Very convincingly so, in some cases. What makes an expert an “expert" is often in the eye of the beholder. Collins and Evans take three case studies, those using color-blind individuals, pitch-blind individuals (those who do not have “perfect-pitch” capabilities, which admittedly is a very large percentage of the population), and Gravitational Wave scientists. In this section, they set up experiments similar to a “Turing Test” (referred to in the book as the "Imitation Game") to see if people who do not have knowledge of a particular area (meaning those who are not color-blind, pitch-perfect, or don’t really know about Gravitational Wave science) can be fooled by people who can “talk a mean game” about a particular area, but don’t really have that experience. Conversely, the same experiment was done with those who did have experience with these areas (those who are color-blind, pitch-perfect and have experience with Gravitational Wave science). The results showed that those who were genuine experts could spot fakes most of the time (though sometimes could be tricked), and were “conned” a lot less often than those who didn’t have background with these areas. 

These case studies set up the remainder of the book, in which we look at a variety of demarcation points as to where we might want to use some greater discrimination when it comes to just how much we trust certain “experts”. Collins and Evans explore a variety of intersections, such as science vs. art, science vs. politics, hard sciences vs. social sciences, and science vs. pseudo-science. In several cases, we as everyday people find the point where we accept “expertise” moves along a continuum from the original producers of knowledge (the hard sciences) to those who consume knowledge (the arts, but also politics, the social sciences and the pseudo-sciences). Why do we give the pundit, the news anchor, the talk show host, or the pop culture critic credence? Why do we trust their “expertise”? What is it based on? In short, are we being conned?

The book closes with an Appendix on the Three Waves of Science, or perhaps better placed, the three waves of scientific inquiry as relates to the Twentieth and Twenty-first centuries. We have moved the pendulum away from the idea that scientists are "rarified creatures" on the level of high priests of the technical sphere (the first wave), to a time of great distrust in science and relativistic views as to their value and relevance (the second wave), and now to a third wave that is less reverential of the first wave, but more skeptical of the claims of the second wave.

I’d encourage any reader of Rethinking Expertise to read the Introduction and this Appendix first, and then read the rest of the book in order. By doing so, the scaffolding of the ideas being presented makes more initial sense, and will potentially prevent the need for a re-read like I needed (though frankly, that reread may prove to be very insightful). This is not a casual Saturday afternoon read (though it may be for some, it certainly wasn’t for me). This is a dense book, and the sheer quantity and textual volume of the footnotes is significant. Rereads will certainly give further clarification and a better feeling for the ideas.

Bottom Line:

Rethinking Expertise meets many objectives. First, it gives a taxonomy to areas of expertise, and helps solidify an understanding as to where on the continuum our understanding comes from, the level to how (and why) we understand what we do. It also helps us identify how our interactions and experiences, along with direct participation in events and activities, all contribute to the level of expertise that we have (or don’t have). What’s more, it helps us to get a handle on the level of expertise others may have, or may not have. It asks us to think critically about those we trust, and what their intentions may be.