Blog: On Scripting

A script, in the general sense, is something that constrains our actions in some way.

In common talk about testing, there’s one fairly specific and narrow sense of the word “script”—a formal sequence of steps that are intended to specify behaviour on the part of some agent—the tester, a program, or a tool. Let’s call that “formal scripting”. In Rapid Software Testing, we also talk about scripts as something more general, in the same kind of way that some psychologists might talk about “behavioural scripts”: things that direct, constrain, or program our behaviour in some way. Scripts of that nature might be formal or informal, explicit or tacit, and we might follow them consciously or unconsciously. Scripts shape the ways in which people behave, influencing what we might expect people to do in a scenario as the action plays out.

As James Bach says in the comments to our blog post Exploratory Testing 3.0, “By ‘script’ we are speaking of any control system or factor that influences your testing and lies outside of your realm of choice (even temporarily). This does not refer only to specific instructions you are given and that you must follow. Your biases script you. Your ignorance scripts you. Your organization’s culture scripts you. The choices you make and never revisit script you.” (my emphasis, there)

When I’m driving to a party out in the country, the list of directions that I got from the host scripts me. Many other things script me too. The starting time of the party—combined with cultural norms that establish whether I should be very prompt or fashionably late—prompts me to leave home at a certain time. The traffic laws and the local driving culture condition my behaviour and my interactions with other people on the road. The marked detour along the route scripts me, as do the weather and the driving conditions. My temperament and my current emotional state script me too. In this more general sense of “scripting”, any activity can become heavily scripted, even if it isn’t written down in a formal way.

Scripts are not universally bad things, of course. They often provide compelling advantages. Scripts can save cognitive effort; the more my behaviour is scripted, the less I have to think, do research, make choices, or get confused. In my driving example, a certain degree of scripting helps me to get where I’m going, to get along with other drivers, and to avoid certain kinds of trouble. Still, if I want to get to the party without harm to myself or other people, I must bring my own agency to the task and stay vigilant, present, and attentive, making conscious and intentional choices. Scripts might influence my choices, and may even help me make better choices, but they should not control me; I must remain in control. Following a script means giving up engagement and responsibility for that part of the action.

From time to time, testing might include formal testing—testing that must be done in a specific way, or to check specific facts. On those occasions, formal scripting—especially the kind of formal script followed by a machine—might be a reasonable approach enabling certain kinds of tasks and managing them successfully. A highly scripted approach could be helpful for rote activities like operating the product following explicitly declared steps and then checking for specific outputs. A highly scripted approach might also enable or extend certain kinds of variation—randomizing data, for example. But there are many other activities in testing: learning about the product, designing a test strategy, interviewing a domain expert, recognizing a new risk, investigating a bug—and dealing with problems in formally scripted activities. In those cases, variability and adaptation are essential, and an overly formal approach is likely to be damaging, time-consuming, or outright impossible. Here’s something else that is almost never formally scripted: the behaviour of normal people using software.

Notice on the one hand that formal testing is, by its nature, highly scripted; most of the time, scripting constrains or even prevents exploration by constraining variation. On the other hand, if you want to make really good decisions about what to test formally, how to test formally, why to test formally, it helps enormously to learn about the product in unscripted and informal ways: conversation, experimentation, investigation… So excellent scripted testing and excellent checking are rooted in exploratory work. They begin with exploratory work and depend on exploratory work. To use language as Harry Collins might, scripted testing is parasitic on exploration.

We say that any testing worthy of the name is fundamentally exploratory. We say that to test a product means to evaluate it by learning about it through experimentation and exploration. To explore a product means to investigate it, to examine it, to create and travel over maps and models of it. Testing includes studying the product, modeling it, questioning it, making inferences about it, operating it, observing it. Testing includes reporting, which itself includes choosing what to report and how to contextualize it. We believe these activities cannot be encoded in explicit procedural scripting in the narrow sense that I mentioned earlier, even though they are all scripted to some degree in the more general sense. Excellent testing—excellent learning—requires us to think and to make choices, which includes thinking about what might be scripting us, and deciding whether to control those scripts or to be controlled by them. We must remain aware of the factors that are scripting us so that we can manage them, taking advantage of them when they help and resisting them when they interfere with our mission.

Want to know more? Learn about Rapid Software Testing classes here.

5 responses to “On Scripting”

  1. Given the above I think that there’s a really interesting point where the scripting of biases, fast thinking, social conditioning and so on meets the agency of critical thinking, slow thinking, scientific thinking and so on.

    There also seems to be a divide between natural scripts (e.g. our pattern-seeking nature) and unnatural scripts (e.g. a computer script, a written checklist).

    And I think it’s very interesting to consider internal scripts that we carry with us vs external scripting artefacts (I think you prefer “artifacts”?) as let’s say a shopping list written on paper is an external script… but when we read it and apply it to our decisions we internalise that script and the complexities of that communication process (from one agent to another where one is human) is one reason why scripted testing isn’t for novices.(http://www.satisfice.com/blog/archives/672). Even the credence we grant to conversations can script us.

    It seems to be that scripts are the gap between objective reality and our interpretation of it – or is that going too far?

    Michael replies: I think it’s going too far to say that scripts are THE gap between our interpretations and “objective” reality (whatever that is). But they could certainly represent A gap.

  2. […] On Scripting Written by: Michael Bolton […]

  3. Kristjan Uba says:

    Wonderful description of how testing consists of many, many different activities. And the importance of them depends on the context (project, timeline, people, etc., etc.).

    But for me, above else, the post illustrates that making conscious decisions about your choices is the most important thing.

    It might be right to only do automated scripted testing, or only relay on end users for doing the beta testing. Or choosing to use ‘boundary testing’ in this way or trying out all combinations instead of using all-pairs.

    As long as you have considered alternatives (and have chosen to be *aware* of them by exploring) you can do whatever makes sense to you (in your context).

    Michael replies: I’d agree, while emphasizing that it would be well to make sure that you’re in sync with the client of your testing.

    And to be aware of options you need to learn about your project, your team, your skills, etc. These activities take effort (and skill) and can’t be scripted. Take the time to learn and you’ll be more effective in the long run.

    Absolutely. Thank you for the comment!

  4. […] On Scripting by Michael Bolton […]

Leave a Reply

Your email address will not be published. Required fields are marked *