One of the Kindly Contributors to the software-testing mailing list, José Alejandro Betancur, writes:
I have been creating the Test Department at a developer company (www.intergrupo.com) for about 1 1/2 year. But I’m facing a problem right now, and i don’t know if it is a normal behavior. Since we are here, the number of issues (bugs) is increasing!!! and that way it has been difficult to stop them to go to production or client certification. Our staff thinks that before the test team the developer team was delivering more stable solutions. I know that as we test, we would inevitable find more and more issues, but it looks to management that our developers are more relax and delivering without self test the solution first.
Anyone that has created a test team has that problem? What kind of metrics do you think I most check?
This is a classic example of a reversal effect. I’m aware of it having happened in several organizations, and I watched it happen myself at Quarterdeck.
Marshall McLuhan talked a lot about media. A lot of people don’t realize that his ideas were different from the ones that we usually associate with the words that he used. To McLuhan, a medium is anything that effects a change—a new tool or technology, an idea, a programming language, a hat—anything that appears as a new figure somewhere in an existing ground.
In order to investigate the impact or significance of a medium (or as McLuhan called it, the medium’s “message”), McLuhan proposed four probes, or questions, that one could ask about a medium: 1) To what extent does this medium enhance or extend or intensify or expand or enable some human capability? 2) What previously obsolete medium (perhaps from a vastly different field or from a long time ago) does this new medium retrieve, or remind us of? 3) What existing (and perhaps ubiquitous) medium does this new medium make obsolete? 4) How might this medium reverse its intended or expected or beneficial effect when it exceeds its capacity, or when it becomes the norm? Every medium, said McLuhan, exhibits the general characteristics that the probes ask about—extension, retrieval, obsolescence, reversal.
The intention is to try to look at a new or anticipated medium using these probes, thereby to determine the medium’s impact, or “message”. When McLuhan said “the medium is the message”, what he meant was that the significance of the medium is in its impact, or its effect upon the existing ground.
(Mark Federman, at http://whatisthemessage.blogspot.com, explains Marshall McLuhan’s Laws of Media elegantly and clearly; “Creating a Culture of Innovation” is the first paper to look for, in the column on the right of the blog page. All the papers are well worth a read. Even better is Mark’s book, co-authored by Derrick deKerckhove, “McLuhan for Managers”, which is hard but still possible to get. It’s a superb book for people involved with innovation, and it’s also a great example of general systems thinking at work.)
So: the test team enhances the ability of the product team to find bugs, to report problems, to understand the product. It retrieves the idea of the test pilots, the food taster, the editor, the lab researcher. It obsolesces, at least to some degree, the field technician, the support centre, the unhappy customer. And it reverses into developers who won’t check their own code, blaming and finger-pointing, mindless confirmation of trivial test ideas, automated behaviour, product managers passing responsibility for release decisions on to the testers…
I arrived at that list after a minute or two of brainstorming; it could be much longer. What would happen if you spent 15 minutes on the list with your test team, or your development team, or your managers? You might be able to set clearer goals (or limits) about what you want to extend, how you want to model the team, what or who the team might make redundant, and (at least to some degree) to anticipate and control for some of the reversal effects of the test team.
One more thing: rather than thinking about metrics, try thinking about observations that you might make. If it makes sense simply to describe the observation, do that. If it makes sense to measure the observation, remember that metrics are media too. That is, each metric you use will extend, retrieve, obsolesce, and reverse some aspect of your observation.
Postscript: I was intending to wait and write a column on this subject, but Bret Pettichord asked me (quite emphatically) to blog it. It’s wonderful to have colleagues who encourage me to publish, and I’m very grateful to Bret for that. I’m also deeply grateful to Mark Federman for several provocative and stimulating conversations in which he has helped me to understand and apply these ideas.