DevelopsenseLogo

What Is A Tester?

A junior tester relates some of the issues she’s encountering in describing her work.

To the people who thinks she “just breaks stuff all day”, here’s what I might reply:

It’s not that I don’t just break stuff; I don’t break stuff at all. The stuff that I’ve given to test is what it is; if it’s broken, it was broken when I got it. If I break anything, consider what my colleague James Bach says: I break dreams; I break the illusion that the software is doing what people want.

And when somebody doesn’t understand what a tester does, these are some of the metaphors upon which I can start a conversation. These are some things that, in my testing work, I am or that I aspire to be.

I’m a research scientist. My field of study is a product that’s in development. I research the product and everything around it to discover things that no one else has noticed so far. An important focus of my research is potential problems that threaten the value of the product. Other people—builders and managers—may know an immense amount about the product, but the majority of their attention is necessarily directed towards trying to make things work, and satisfaction about things that appear to work already. As a scientist, I’m attempting to falsify the theory that everything is okay with the product. So I study the technologies on which the product is built. I model the tasks and the problem space that the product is intended to address. I analyze each feature in the product, looking for problems in the way it was designed. I experiment with each part of the product, trying to disprove the theory that it will behave reasonably no matter what people throw at it. I recognize the difference between an experiment (investigating whether something works) and a demonstration (showing that something can work).

I’m an explorer. I start with a fuzzy idea of the product, and a large, empty notebook. I treat the product as a set of territories to be investigated, a country or city or landscape to learn about. I move through the space, sometimes following a safe route, and sometimes deviating from the usual path, and sometimes going to extremes. I might follow some of the same paths over and over again, but when I really want to learn about the territory, I turn off the marked roads, bushwhacking, branching and backtracking, getting lost sometimes, but always trying to see the landscape from new angles. I observe and reflect as I go. In my notebook I create pages of maps, diagrams, lists, journal entries, tables, photos, procedures. Mind you, I know that the book is only a pale representation of what I’ve seen and what I’ve learned, no matter how much I write and illustrate. I also know that many of the pages in the book are for myself, and that I’ll only show a few pages to others. The notebook is not the story of my exploration; it helps me tell the story of my exploration. (Here’s some more on notebooks.)

I’m a social scientist. I’m a sociologist and anthropologist, studying how people live and work; how they organize and interact; how things happen in their culture; and how the product will help them get things done. That’s because a product is not merely machinery and some code to make it work. A product fits into society, to fulfill a social purpose of some kind, and humans must repair the differences between what machines and humans can do. Thus testing requires a complex social judgement—which is much more than a matter of making sure that the wheels spin right. (I am indebted to Harry Collins for putting this idea so clearly.) What I’m doing has hard-science elements (just as anthropology has a strong biological component), but social sciences don’t always return hard answers. Instead, they provide “partial answers that might be useful”. (I am indebted to Cem Kaner for putting this idea so clearly.) As a social scientist, I strive to become aware of my biases so that I can manage them, thereby addressing certain threats to the validity of my research. So, I use and interact with the product in ways that represent actual customers’ behaviour, to discover problems that I and everyone else might have missed otherwise. I gather facts about the product; how it fits into the tasks that users perform with it, and how people might have to adapt themselves to handle the things that the product doesn’t do so well.

I’m a tool user. I’m always interacting with hardware, software, and other contrivances that help me to get things done. I use tools as media in the McLuhan sense: tools extend, enhance, intensify, enable, accelerate, amplify my capabilities. Tools can help me set systems up, generate data, and see things that might otherwise be harder to see. Tools can help me to sort and search through data. Tools can help me to produce results that I can compare to my product’s results. Tools can check to help me see what’s there and what might be missing. Tools can help me to feed input to the product, to control it, and to observe its output. Tools can help me with record-keeping and reporting. Sometimes the tools I’ve got aren’t up to the task at hand, so I use tools to help me build tools—whereupon I am also a tool builder. I’m aware of another aspect of McLuhan’s ideas about media: when extended beyond their original or intended capacity, tools reverse into producing the opposite of their original or intended effects.

I’m a critic. Like my favourite film critics, I study the work and how it might appeal—or not—to the audience for which it is intended. I study the technical aspects of the product, just as a film critic looks at lighting, framing of the shots, and other aspects of cinematography; at sound; at editing; at story construction; and so forth. I study culture and history—I study the culture and history of software—as a critic studies those of film—and of societies generally—to evaluate how well the product (story) fits in relation to its culture and its period and the genre in which the work fits. I might like the work or not, but as a critic, my personal preferences aren’t as important as analyzing the work on behalf of an audience. To do this well, I must recognize my preferences and my biases, and manage them. I fit all those things and more into an account that helps a potential audience decide whether they’ll like it or not. (A key difference is that the reader of my review is not the audience of a finished product; my review is for the cast, crew, and producers as the product is being built.)

I’m an investigative reporter. My beat is the product and everything and everyone around it. I ask the who, what, where, when, why, and how questions that reporters ask, and I’m continually figuring out and refining the next set of questions I need to ask. I’m interacting with the product myself, to learn all I can about it. I’m interviewing people who are asking for it, the people are who building it, and other people who might use it. I’m telling a story about what I discover, one that leads with a headline, begins with a summary overview and delves into to more detail. My story might be illustrated with charts, tables, and pictures. My story is truthful, but I realize the existence of different truths for different people, so I’m also prepared to bring several perspectives to the story.

There are other metaphors, of course. These are the prominent ones for me. What other ones can you see in your own work?

10 replies to “What Is A Tester?”

  1. This is great, Michael; thanks for sharing.

    A metaphor that I see a lot is that of editor (I’ve written about this before, but on a different tangent: http://testersnotebook.jeremywenisch.com/2014/04/14/testing-and-editing-an-analogy-analysis/). I sometimes think of the developer as a novelist, whose primary focus is getting their large piece of writing fit together and working. As an editor, I help shed light on problems they’ve overlooked and give voice to their potential audience. Just as an editor helps the novelist find plot holes, character inconsistencies, and awkward shifts in tone, I help the developer find security holes, functional inconsistencies, and awkward shifts in UI. The novelist knows exactly what story they intend to tell; as the editor, I strive to understand exactly what story has actually been told in the current draft.

    Michael replies: I’ve used that metaphor too. You’ve given a terrific version of it here.

    Reply
  2. This is a really nice collection.

    A metaphor we sometimes use at our company is the tester as a user advocate, i.e. being the user’s voice (or the voice of reason?) within development, which is often enough lost in the process. I guess that goes somewhere along the lines of social scientist and critic, but I like it nevertheless.

    Michael replies: I recognize the intention there, but I’d advise being careful too. The product owner could (and should) quite reasonably claim to be an advocate for the user. The designer of the product and the designer of the user interface could make that claim too. Programmers may have entirely valid reasons to program things that they believe are in the user’s best interest. In fact, is there anyone on the team who would say that they’re not an advocate for the user?

    I’d put it like this: a part of the tester’s role might be to remind the rest of the team that we’re all working for the user, if (and maybe only if) anyone seems to be losing track of that.

    Reply
  3. Imagine this building. Part of it is ready and there seem to be people living there or doing all kinds of things. Part is being renewed and expanded as we speak. The house is an establishment that provides a kind of service to people. So naturally there are persons involved that have very different interests depending on the situation they’re in.

    One disturbing thing is, that the house is said to be standing on an old burial ground and there are people who believe it’s haunted. It’s one theory, that people have, why the construction work of the new parts only very slowly advances, why parts of the old building keep cracking, creaking and breaking and why even some of the people have reportedly been killed while being in the house.

    It’s one theory.

    I am the private detective they hired to sort out that mess – my client wants to know if there is any truth in these allegations. My task in this case is to interview witnesses, speak to my informants, try to collect and preserve evidence. My mission is to unearth information, so that my client may make a more informed decision in the stated matter. That’s all I am allowed to say in this case; for now.

    Tomorrow I may have a different case with different tasks and different nested missions.

    Just as a private detective doesn’t enforce the law and provides the client, who may be an individual or a group, merely investigatory services, I as a software tester don’t enforce decisions about “the stated matter”. I serve my client/-s and aim to provide them investigatory services to attend to their desires and needs.

    A software project may be one big case or a collection of several cases, but in spite of that I have to sort out that mess.

    —-

    This is a metaphor that I like – the thought of being a private detective.

    Thank you for you thought provoking blog post, Mr. Bolton!
    Isn’t it interesting how similar the skills of a tester are to the skills of seemingly totally different roles.

    Michael replies: I’ve never liked the police detective analogy, but I really like the private investigator one, and you’ve done a nice job of setting it out here. The most important part to me is that, precisely as you say, testers don’t enforce the law, but they do provide information that people can use as they wish.

    Reply
  4. Another metaphor is “Tester is a therapist”.. House M.D. in best scenario =) Someone who listens to people but always keeps in mind that everybody lies. Someone who performs differential diagnosis to find the root cause of a “disease”. NOT someone who gives you the pills just in case.
    In general tester-therapist gathers the anamnesis about the patient (questioning and listening), performs medical tests, uses instruments e.g. to calculate the number of leukocytes and finally provides the health recommendations. Tester-therapist does not perform ALL available tests. He follows the path using his knowledge, experience and skills. Real tester-therapist knows that some diseases are incurable. And so on =)
    PS: Thank you Michael for your posts. These notes, yours and Mr. Bach’s, revived a tester in me.

    Michael replies: Thank you, and you’re welcome. I liked House too, especially in the early days.y

    Reply

Leave a Comment