Until today, my reading of Mary Reilly’s Play As Exploratory Learning had been limited to occasional stolen glances into Cem Kaner’s library, but a copy of this out-of-print book arrived today. Browsing (that is, a little exploratory reading) yielded this (from Chapter 3, “An Explanation of Play”, page 117):

“The pursuit of the rumored goodness and usefulness that play might have for man is plagued by the difficulties inherent in the processes of exploration. One of the first problems is the very obviousness of play. It is a behaviour endlessly in plain sight, and because it is a behaviour there in plain sight it lacks the intrigue that the unknown has for scientists. Intellectuals go to any lengths to avoid the obvious and theorists in particular disdain it. The real difficulty that confronts any broadened explanation of play is that it is an obvious commonplace behaviour breeding contempt dampening investigatory interest.”

Well, ain’t that just like exploratory approaches to testing? (For “play” read “exploratory testing”; for “scientists” and “theorists”, read “many programmers”; and for “intellectuals” read “traditional testing theorists”.) Explaining exploratory testing is hard because we appear to be simply “trying the program to see if it works”. What’s not obvious to the untrained observer is the stuff that’s going on behind the eyeballs: modeling the test space; determining coverage, oracles, and procedures; and making decisions about how to configure, operate, observe, and evaluate the product. It would be nice if other people could see all that stuff, but we can’t. So, as a community, those of us who recognize the value of exploratory approaches—that is, the fact that they’re central to excellent testing—are going to have to keep talking about them.

Jason Gorman is an interesting guy, and has a lot to say. I agree with lots of it, especially his iconoclastic position on agilism. This time, I’d like to disagree with two paragraphs in a recent blog entry. The second one first.

But I suspect in 5-10 years’ time, as test-driven development becomes more popular and teams become more ambitious in their testing efforts, test developers will be in great demand and will be able to command high salaries. I see them becoming as important as architects are viewed as today. Maybe more so, since they actually add value on a project 😉 (Only kidding)

This seems to have been coming up a lot, so I’ll say it here: I’m agnostic about architects, but testers don’t add value to a project; as James Bach suggests, testers help to defend the value that’s already there, or help to identify ways in which value may be lacking. Testers raise questions and make observations; the people who make decisions based on those observations are the ones who add value. We help them do that, but we don’t do it intrinsically on our own.

Realistically, you must be a developer before you can become a test developer, since learning how to create high quality code takes longer to master than learning how to write effective tests. That’s not to denegrate the discipline of software testing: anyone who’s read Robert Binder’s book on Testing Object Oriented Systems will know that, if you wanted to, you could dedicate a lifetime to understanding testing. But, let’s be honest now, it takes longer to learn how to code from scratch than it does to learn how to break code*. (* Is that my inbox I can hear filling up again?)

Well, first, testers don’t break code; the code was already broken when it showed up. (I think I heard this first from Alan Jorgenson.) That’s a fairly trivial objection, though.

The more serious problem is with the assertion that “learning how to create high quality code takes longer to master than learning how to write effective tests”. Testing is not merely a matter of writing test code. Testing is about questioning the value of a product and the potential threats to that value. It takes the better part of a lifetime, I think, to understand value. Every time I think I have a handle on that, someone or something comes along to teach me another lesson in humility. But until we’re sure we’re able to ask the right questions about value, I’ll contend that we can’t know the right answers to whether our code is of high quality or not. By that reckoning, let’s be honest now: learning how to create high-quality code and learning how to question should be inseparable. It’s a rare person who can do either one very well, and even an even rarer person that can do both very well. That’s why we need developers and testers—and why we tend to need a few of each.

Here’s a heuristic: when someone is describing (or, especially, dissing) some practice or methodology, don’t bother taking them seriously unless they know the difference between tenants and tenets. Examples abound.

Over the last few months, I’ve been wrestling with a book called Sensemaking in Organizations, by Karl Weick. I’ve got bogged down in it from time to time, but it’s fascinating. Weick describes sensemaking as having seven properties:

1. it’s grounded in constructing or enhancing the identity of an individual or group;
2. it’s retrospective, or based on “meaningful lived experience”;
3. it’s “enactive of sensible environments”, which is kind of circular; it means that part of the process of sensemaking involves trying to produce an environment in which further sensemaking is possible;
4. it’s a social process; it happens in the presence of others, or with the knowledge that others will understand, approve, or be involved;
5. it’s ongoing; despite the fact that it’s retrospective, it doesn’t have a clear starting point either, because “people are always in the middle of things”;
6. it’s based on extracted cues, “simple, familiar structures that are seeds from which people develop a larger sense of what may be occurring”; and
7. driven by plausibility rather than accuracy, which means to me that sensemaking is a heuristic process.

The book is rewarding, and I recommend it, but there is one story that on its own makes the book worth the price of admission. Over the last few months, I’ve treated a couple of newsgroups to the story, which I had heard before from Jerry Weinberg as an example of the heuristic, “When the map and the territory disagree, believe the territory.” But Weick’s analysis adds some extra richness that speaks to the idea of reasonable limits on planning and increased emphasis on doing.

Paraphrased, the story is that an Hungarian Army unit is on patrol in the Swiss Alps. A big snow storm comes up, and they don’t come back to camp for a day, two days, three days. Their lieutenant is now panicked, thinking that he has sent these men to their deaths… and then the unit walks back into camp. “Wow! We thought you were lost for good–where have you been?” “Well, when the storm came up, we hunkered down, and when we finally poked our heads out, everything was covered with snow and we realized that we were lost. One of us had a map, though, so we opened it up, and we realized that if we went down the hill we’d hit a river, and if we followed the river we’d hit the town, and here we are.” The lieutenant looked at the map and realized that it wasn’t a map of the Alps, but of the Pyrenees.

Says Weick (on page 55), “this incident raises the intriguing possibility that when you are lost, any old map will do. For example, extended to the issue of strategy, maybe when you are confused, any old strategic plan will do. Strategic plans are a lot like maps. They animate and orient people. Once people begin to act (enactment), they generate tangible outcomes (cues) in some context (social), and this helps them discover (retrospect) what is occurring (ongoing), what needs to be explained (plausibility), and what should be done next (identity enhancement). Managers keep forgetting that it is what they do, not what they plan, that explains their success. They keep giving credit to the wrong thing–namely, the plan–and having made this error, they then spend more time planning and less time acting. They are astonished when more planning improves nothing.”