DevelopsenseLogo

EuroSTAR Trip Report, Part 1

Way way back in 2003, Bret Pettichord first published a paper on schools of software testing. The paper was controversial. Some people found it helpful to identify different schools of thought, for the purpose of understanding ways in which reasonable people might disagree reasonably.  Others found even the mention of disagreements within the field to be distasteful and divisive.  Some people identified with particular schools. Others, sometimes indignantly, refused to be pigeonholed. Yet it’s clear that in any field of endeavour, including testing, there are always communities of thought and practice. Sometimes those communities are isolated; sometimes there are trading zones between them.

No matter how one might label the communities, two broad categories were apparent to me at this year’s EuroSTAR conference. One group seems to focus on testing in terms of confirmation, verification, validation, quality assurance; getting the right answers to prescribed questions; checking. This group’s approach includes a strong focus on artifacts—requirement documents, detailed test plans, and scripted test cases. This group (let’s call it the Traditionalists) also seems to focus on processes and tools, on negotiated contracts, and on following plans—items on the right side of the Agile Manifesto. I don’t claim membership in the Agile School. Although though I greatly admire the principles in the Manifesto, for me, the first thing to look at is the project’s context, and to proceed accordingly. The Traditionalistas, as I see it, emphasize the Agile Manifesto’s “things on the right”. Probably they do so with the desire to dispel variability, subjectivity, and unpredictability from testing.  I try to be empathetic towards those who advocate the things on the right, since those aren’t unreasonable things to want; it’s just unreasonable, in my view, to believe they’re the more important things in the complex, messy, human, and constantly changing world of software development.

The other, significantly smaller—and, in general, younger—group that I observed at EuroSTAR sees testing as questioning, exploration, discovery, investigation, and learning—and quality assistance. Let’s call that group the Vanguard. The Vanguard realizes that getting the right answers is important, but asking the right questions is more important—and recognizing that today’s “right questions” today are probably different from yesterday’s “right questions” is more important still. In broad strokes, the Vanguard prefers

experience reports over “best practice” talks
conversation over lectures
hands-on exercises over PowerPoint presentations
tools for investigation over tools for confirmation
dialogue over monologue
sitting in a circle over classroom format
finding things out over hearing the answer

And, as in the Agile Manifesto, they recognize value in the things on the right, but they value the things on the left more.

The Vanguardistas are eager to participate in testing exercises, and to exchange testing skills by example and by dialog. The Vanguard raises some difficulties for traditional trainers and presenters, because the Vanguard tends to want to ask questions and challenge authority—and as a trainer and presenter, I think that’s great. Many of the Vanguardistas participate in or organize Weekend Testing sessions. Almost all of them are on Twitter. They want to revive and reinvent testing as a sophisticated art that requires vigourous critical thinking. They’re indefatigably curious and engaged, and they’re becoming recognized as leaders in their community and in the testing craft.

One hallmark of the Vanguard at EuroSTAR was that they gravitated towards doing testing in the Test Lab, once again run by James Lyndsay (@workroomprds on Twitter) and Bart Knaack (@Btknaack on Twitter) after their impressive success at EuroSTAR last year. This year, 180 people visited the Test Lab. Though probably a minority, that’s a significant percentage of the overall attendees, and is all the more remarkable because, for space reasons, the Test Lab was quite a distance away from most of the presentations. This year there were more applications to test, more sharply focused vendor presentations, specific guidance for those who needed it, and lots of pairing and sharing. For me, one of the more memorable events was a relatively impromptu exploratory testing management roundtable, facilitated by James, with more than 20 people attending—remarkable because the event wasn’t noted specifically as a scheduled part of the conference programme; it was set up in the Test Lab, advertised by word of mouth, and fundamentally collaborative. The roundtable was one of those things that put the confer back in conference.

Of many high points of the roundtable conversation, the big one for me was the group’s recognition that testers don’t need to be domain experts from the outset of a testing assignment. Instead, testers can partner with domain experts in review and hands-on testing sessions, and in that collaboration get some excellent testing work done immediately. An exploratory testing cycle—test design, test execution, test result interpretation, learning, debriefing—drives rapid and highly effective learning about the domain. As Rob Sabourin (more on Rob later) articulated it: “Here’s a beautiful charter for a test session: Sit with a customer/user and ask ‘What gets in the way of you doing your work?'”

James and Bart were assisted this year by the Test Lab apprentices, Henrik Emilsson (@henrikemilsson on Twitter) and Martin Jansson (@martin_jansson on Twitter). At EuroSTAR 2011, management of the Test Lab will pass to Henrik and Martin. It’s in good hands. Henrik and Martin are members of a blogging cabal called thoughts from the test eye, which has been producing incisive, thoughtful reflections on testing since February 2008. An outstanding example is a blog post announcing their own list of software quality characteristics, in which they build on one of the pillars of James Bach‘s Heuristic Test Strategy Model. But that’s just one example. Read the back issues and put the new ones in your feed reader.

Another member of the test eye collaborative is Rikard Edgren. Rikard was one of the conference chairs of EuroSTAR this year. He seems to have found a way to violate some fundamental law of physics by being everywhere at the same time; whenever I turned around, he was there with an expression on his face that reflected his keen observational skill and his sly humour. I’ve been lucky to have many interesting chats with him, not only this year but in years previous.

More on EuroSTAR 2010 tomorrow.

7 replies to “EuroSTAR Trip Report, Part 1”

  1. Michael, good report.

    The Test Lab was indeed the hot-spot at EuroStar. Whenever I wasn’t sure if one of the lectures would be worth it I headed to the lab. Avoiding risk by picking an option that would certainly deliver value :). I paired with people I had never met before. Fantastic.

    However, I did NOT feel such accentuated division between the two communities you mention.

    Michael replies: I’m glad to hear that. Certainly my impression was influenced by my own selection bias. Some of the other people I respect felt the same kind of division that I did; others didn’t. More than anything else, our respective observations emphasize that different people will have different experiences, even in “the same” situation.

    On the contrary, when speaking with people, I noticed a big respect for exploratory tests, context questions and learning. So big a respect that sometimes it was expressed as ‘fear’ of not being able to do it right.

    So if once upon a time the focus on “requirement documents, detailed test plans, and scripted test cases” was due to a belief that it’s a proper way to work, maybe nowadays this focus is more due to not yet figuring out how to move out of it—or perhaps due to not finding the courage to try (and fail).

    For me, it means the division is easy to bridge, because there’s willingness.

    Could be. The good news on that front is that there were several talks at the conference that would help with those issues. (More on that in later posts.)

    And on a last note, I have to agree that Rikard was indeed everywhere. I had the pleasure of working with him during your framing session, and he would disappear and reappear suddenly—it felt like Batman and Commissioner Gordon. He is a remarkable person.

    Thanks for the extra perspective, Shmuel.

    Reply
  2. Great report, I’m looking forward to reading more.

    I wasn’t at EuroStar, so obviously I can’t say anything about discussions on the conference, but I tend to agree more with Michael on this one. Test management in the industry is in my experience still far too often focusing on the ‘old school approaches’. And producing a lot of waste because of this. And I don’t see it changing.

    We really need to get better at selling the ‘new school’ and quality assistance (as oppossed to ‘assurance’) to managers.

    We have to appreciate that they need to feel that they’re in control. They’re looking for safety and quality. That’s what we can help them get, but then how do we make them buy what they need instead of what they want?

    Reply
  3. I’m hoping to do a tutorial at EuroSTAR 2011 about being at several places at the same time. And perhaps include teleportation, if I can make the exercises reliable!

    I don’t think the division is that great.

    Michael replies: I’d like to believe that.

    It sure exists, but many of the most important insights seems shared:
    * You need multiple information sources, also outside explicit requirements
    * We don’t have complete knowledge, so exploring is necessary
    * Testing is difficult, and requires skill and collaboration

    I agree that most people seem to share these observations. Perhaps the division is over the hows more than the whats.

    Reply
  4. Michael,

    Interesting observations. I noticed, subconsciously, a similar thing at SIGiST yesterday. The thing that struck me particularly strongly was the fact that the more exploratory crowd (probably that should read “exploratory few” actually) were very aware of the things on the right-hand side of the Agile Manifesto and could see where these things could be valuable in certain contexts but the traditionalists seemed much less aware of the value of the things on the left.

    Self-education also seems much more second-nature to exploratory types and that is why I think there would be a gravitation towards the test lab from such people.

    Thanks for the write-up. I hope to share my report of SIGiST later today but it might end up being over the weekend.

    Regards,

    Stephen

    Reply
  5. Do notice that “the Vanguard” the great underground movement of passionate and motivated testing individuals you have listed in the trip reports are now embraced by the EuroStar team and included as online ambassadors etc.

    Reply

Leave a Comment