For almost as long as I’ve been a tester, with occasional lapses into process enthusiasm, I’ve been questioning the value of test automation as a presumed good, especially when the automation is deployed against the highest levels of the application. Automation is a tool, and there is great value in tools. But with that value comes risk.
The Agile Manifesto, properly in my view, emphasizes individuals and interactions over processes and tools, and it emphasizes working software over comprehensive documentation. The Manifesto notes that “while there is value in the things on the right, we value things on the left more.”
McLuhan had some remarkable observations on tools, which he considered a subset of media. I wrote about the value of McLuhan thinking for testers here. McLuhan famously identified writing—in particular the phonetic alphabet—as a technology. In his Laws of Media, he points out that one of the effects of a medium is to extend or enhance or enable or accelerate or intensify some human capability in some way. Another effect occurs when the medium is stretched or “overheated” beyond its original or intended capacity; it reverses into the opposite of its enabling or extending effect.
I was introduced to McLuhan’s work largely through a CBC Radio program called Ideas. In 1988, David Cayley covered a conference on Orality and Literacy, co-sponsored by the McLuhan Program at the University of Toronto and the Toronto Semotics Circle (for a description of the program and of the conference, go here, and then search for “literacy”). The conference was set up to question the idea of literacy as the centre of education—not to reject it, but to question it.
The motivation behind the questioning was to understand better the role of literacy in education and in the world in general. Some scholars pointed out that literacy has to be seen in its human context, as an extension of oral discourse, because it is as listeners and speakers that we evolved, not as readers and writers. As David Pattanayak, the director of the Central Institute of Indian Languages at the University of Mysore put it, “What I am worried about is that there are 800 million illiterates in the world, and for those 800 million illiterates, there is nobody to speak. We are speaking as though literacy is responsible for everything—for family welfare, GNP increase, for modernization, for all kinds of things, but I don’t think that is correct…The whole question is that there are illiterates and there are literates, and we should be looking for interaction among the illiterates and the literates, rather than trying to prove the superiority of one over the other.” Why? Because literacy is one means to an end; it helps with many things, but it certainly does not guarantee the accomplishment of our human goals. As he later goes on to say, “Literacy without social concern is meaningless.”
To me, this has a direct connection to our business and to our fascination with written and/or automated tests. For many years, we’ve tried to improve testing by trying as hard as possible to remove the messy human bits from it. James Bach describes this in his essay in The Gift of Time. We’ll call it the Chapel Hill approach to software testing; papers on it are collected in Hetzel’s Program Test Methods (something much closer to being the first book on software testing than Myers’ The Art of Software Testing, by the way). The Chapel Hill approach to reducing testing’s problems with those unreliable humans, says Hetzel, is to lean hard on media, improving “written specification methods” and using “unambiguous testable specification languages”, rather than treating testing as an open-ended investigative process.
A bunch of us, led by the work of Jerry Weinberg, Cem Kaner, and James have been questioning the value of putting media at the centre of testing for qute some time now. Questioning the value of written artifacts isn’t exactly new; it goes back still farther than that. How far? How about the Greek philosophers—Plato, and Socrates?
There’s a version of Plato’s dialogue Phaedrus online. The part that most interests me starts with the paragraph that you can find by bringing up the link and using your browser’s search function to look for “Theuth”. Read that paragraph and a little further, and you come across this, when Socrates talks of written-down speeches:
You would imagine that they had intelligence, but if you want to know anything and put a question to one of them, the speaker always gives one unvarying answer. And when they have been once written down they are tumbled about anywhere among those who may or may not understand them, and know not to whom they should reply, to whom not…”
Wait, it gets better. I recommend reading this bit slowly, savouring it:
Until a man knows the truth of the several particulars of which he is writing or speaking, and is able to define them as they are, and having defined them again to divide them until they can be no longer divided, and until in like manner he is able to discern the nature of the soul, and discover the different modes of discourse which are adapted to different natures, and to arrange and dispose them in such a way that the simple form of speech may be addressed to the simpler nature, and the complex and composite to the more complex nature-until he has accomplished all this, he will be unable to handle arguments according to rules of art, as far as their nature allows them to be subjected to art, either for the purpose of teaching or persuading;-such is the view which is implied in the whole preceding argument.
My emphasis—context-driven thinking! Wait; it gets better yet. Really slowly, now:
But he who thinks that in the written word there is necessarily much which is not serious, and that neither poetry nor prose, spoken or written, is of any great value, if, like the compositions of the rhapsodes, they are only recited in order to be believed, and not with any view to criticism or instruction; and who thinks that even the best of writings are but a reminiscence of what we know, and that only in principles of justice and goodness and nobility taught and communicated orally for the sake of instruction and graven in the soul, which is the true way of writing, is there clearness and perfection and seriousness, and that such principles are a man’s own and his legitimate offspring;-being, in the first place, the word which he finds in his own bosom; secondly, the brethren and descendants and relations of his others;-and who cares for them and no others-this is the right sort of man; and you and I, Phaedrus, would pray that we may become like him. (My emphasis added all over the place, there.)
What happens when we apply all this to testing? To me, this says
- that conversation, rather than documentation, is central to the work that we do;
- that notions of correctness are pointless unless they’re based on value;
- that we need to study and practice testing, or anything else that we wish to understand;
- that we must be cautious and think critically;
- that the focus on nomenclature and unquestioned bodies of knowledge proffered by the ISTQB and other certifiers is foolish; and
- that we should aspire to the values that Socrates proposes.
To me it also says that if we want to be great testers, it would be a good idea to study philosophy, focusing (as James suggests)(James Bach, not William James, although he’d probably agree) on ontology (how we conceive of the world, the nature of being) and epistemology (how we know what we know).
And, yes, there is irony too. Here’s Plato, griping through Socrates about how dangerous writing is, and he’s written down this dialogue. What does that tell us? To me, it suggests that everything that we have to say about what we think and what we do is based, not on absolute principles of truth and certainty, but on heuristics and skepticism.
What does this say to you?