DevelopsenseLogo

Oracles from the Inside Out, Part 1: Introduction

In Rapid Testing (and, we believe, in all worthwhile testing) learning about the product and finding problems that matter are central to the mission. In the role of a tester, I support the business and development team by finding and describing things I believe to be important problems that might threaten the value of the product.

A skilled tester should be able do that at any stage—as soon as there’s something to test—with any kind of product, whether it’s a compiled program, a component, a function, source code, a specification, a requirements document, a feature idea, a model, a diagram… Any of these products may have observable, describable problems associated with them.

To be effective, I must be able to provide a credible description of why I believe those problems to be problems that matter. The goal is to give people information that helps them to decide whether the product they’ve got is the product they want. Anyone who is a consumer of that information is a client of the service that I offer.

As a tester, I’m neither the developer nor the product owner. I don’t have authority over the product, the project, or the business. It is not part of the testing role to fix the problem (that rests with the developers); it is not a tester’s responsibility to decide that the problem shall be fixed, nor when, nor how (that rests with the developers and the managers); nor is it a tester’s role to make decisions about the scope of the product (that too rests with others).

I don’t even have authority to declare definitively that something is a problem. For some problems, designers and developers make that determination. A higher authority rests within a business role—the person responsible for decisions about developing and releasing the product, like a project manager or product manager.

Some testers are uncomfortable with the reality of these limitations. To me, that would be like an investigative reporter being upset that he can’t stamp out malfeasance, prosecute corrupt politicians, and restructure local government. Investigative reporters investigate to reveal problems and issues. That’s an honourable and important job on its own—and it requires a specialized set of skills.

When an investigative reporter yearns to change things to be the way he wants them, he becomes a politician. Politicians need skill sets, mindsets, and priorities that are different from those held by reporters. Politicians have certain kinds of authority that investigative reporters don’t have. Politicians can—and perhaps should—take heed of the information that investigative reporters provide, but being a politician means that you’re entitled to ignore that information.

Similarly, being a tester is an honourable and important job, requiring a specialized mindset and a specialized set of skills. Building a quality product is a goal that everyone shares, but the priorities of developers and managers may be different from yours. If you want to be the one making decisions about the product, become a designer, a programmer, or a project manager.

When I encounter something that looks like a problem, I want my clients to understand the nature of the problem as I perceive it. In Rapid Testing, we say that an oracle is a means by which we recognize a problem that we encounter during testing. Yet a couple of years ago, we began to notice another role that oracles play in the social dynamics of software testing. Oracles help us to describe and articulate the problems that we see; and oracles calibrate our feelings about those problems with the rest of the team.

A few years ago, James and I developed this chart to help testers to illustrate the relationships between things as we process our recognition of a problem.

Recognizing a problem tends to start with a feeling, and tends to end with an understanding of the problem and feelings about it that are shared between the tester and the other members of the project community. How does that play out in practice? That’s going to be the subject of some posts over the next little while. Let’s go to the next one.

7 replies to “Oracles from the Inside Out, Part 1: Introduction”

  1. Are you confusing products with Oracles?

    Michael replies: No. But it may look that way if you have a narrower view of oracles and products than we take in Rapid Software Testing. To us, an oracle is “a means by which we recognize a problem when we encounter one in testing”, and a product can anything that has been produced by some person. An oracle is not just a document or a test tool or a comparable product, as I’ll relate in subsequent posts. In today’s post, for example, I show how a feeling can be an oracle (although that is by no means all there is to it).

    You talk about finding problems that affect the value of ‘the’ product. Then you refer to testing all the things that document and model how the product should be as products we test.

    Yes. All the things that document and model the product are products themselves; they have been produced by people. As products, they can be tested. Indeed, if they’re important in helping us determine the presence of the product, they should be tested.

    In these cases it seems like we are testing Oracles and not products? What Oracles do we use to test Oracles?

    Instead of thinking in terms of testing “oracles and not products”, try thinking in terms of testing oracles and products. That’s the subject of this series of posts, and there’s quite a bit more to come. Thank you for the comments and the questions; they’ll help me to test and refine future posts.

    Reply
  2. […] When the developer in our example was unable to determine whether the issue was an actual problem, the fallback was to ask the product owner. The product owner will likely have a look at the issue and decide whether it is a problem or not. This is a classic example of the product owner acting as an oracle [8]. In the Rapid Software Testing namespace an oracle is defined as [7]: […]

    Reply

Leave a Comment