Blog Posts from December, 2010

EuroSTAR Trip Report, Part 3

Sunday, December 12th, 2010

In the last posting, I remarked on some of the people with whom I chatted with at EuroSTAR, and whom I’m seeing as emerging leaders in a community of skilled testers. Here are a few more.

Lynn McKee (Twitter: @lynn_mckee on Twitter) gave an inspiring and very well-attended talk on how to instill passion in testers—and in how to respect and defend the passion that’s there. Lynn walks her talk; her own passion is contagious. She’s on the board of the Association for Software Testing, she’s one of the organizers of POST (a peer conference in Calgary), and she’s one of the organizers of the North American branch of Weekend Testing.  With her colleague, Nancy Kelln (@nkelln on Twitter, also one to watch), Lynn is organizing a session of Rapid Software Testing in Calgary, Alberta that I’ll be presenting in February 2011.

Zeger Van Hese (@TestSideStory on Twitter) was another of the Vanguard’s roving reporters, tweeting up a storm wherever he went with wit and skepticism.  Note that skepticism, as James Bach puts it, is not the rejection of belief, but the rejection of certainty. Zeger also has a terrific blog that I can heartily recommend. The post “Exploring Rapid Reporter” is an exemplary account of what goes through a tester’s mind in the midst of exploration. In his most recent posting, as of this writing, he’s saved me considerable time and effort by providing an excellent report of the Danish Alliance meetup. He links to Shmuel Gershon’s videos of the lightning talks, too; check them out.

It’s always good to have a local agent, and Jesper Ottosen (@jlottosen on Twitter) was Our Man in Copenhagen (along with the aforementioned Carsten Feilberg). All of us who attended the Danish Alliance meeting owes him a vote of thanks for energetically helping to promote and organize it (he said that even that was a learning experience). He also gave a lightning talk that reminded us to look for perfects as well as defects to contextualize our problem reports and to give people recognition for their good work. For so many people, compliments—supported by a visible token—matter! Jesper organized a post-Gala-dinner pub crawl and thereby helped to enable the ensuing extended conversations. Other than his sharp observations on Twitter, I’ve not been familiar with Jesper’s work, which is a problem that I’m looking forward to rectifying. One minor complication: I might have to learn Danish…

Andy Glover, the Cartoon Tester (@CartoonTester on Twitter), was one of the people that I met for the first time after admiring his work from afar. There are many ways to tell the story of testing, and cartooning can be a great way to do it; see Andy’s blog, and Rob Sabourin‘s I Am a Bug (in book and web-based versions) for wonderful examples. Andy led an interesting challenge on Tuesday night, in which he encouraged people to draw their impressions of one of James Bach‘s descriptions of testing—”the infinite art of comparing the invisible to the ambiguous to prevent the unthinkable from happening to the anonymous“. Go ahead; draw that! As a bonus, Andy sold some of his cartoons at the conference and raised a significant sum for charity.

Joris Meerts (@testingref) is someone I met only briefly. I wish we’d had more time to talk. He’s attempting to create a comprehensive historical timeline of the testing craft. I was skeptical when I first looked at Joris’ timeline; it didn’t seem to be missing a number of touchstones. One reason lies in the fact our craft doesn’t have a very good sense of history, and to my knowledge, no one has really attempted to capture it as Joris has. By the same token, it’s also a tricky problem to filter information, because so many important ideas about testing come from other disciplines. Since the conference, Joris and I have begun an email chat on the subject, and it’s clear to me that Joris is going about this very thoughtfully. I intend to do what I can to help him out, and I hope you will too.

Nathalie Rooseboom DeVries (Twitter: @FunTESTic) was another member of the conference committee. One of her roles at the conference, so it seemed, was to question and challenge the Vanguard. To some, that might look like defense of the Traditional way. To me, it looked more like a challenge to the Vanguard to test its own beliefs and practices—which is a very Vanguard thing to to. By posing challenging questions to what people (including me) were saying and tweeting, Nathalie evinced exemplary behaviour for our community. Good for her.

Petter Mattsson wasn’t presenting at EuroSTAR this year, although he has done so in the past. But he was present, and it was lovely to talk to him again. Petter is one of the senior members of the Vanguard, having delivered an experience report on exploratory testing at EuroSTAR several years before it was fashionable to do so. With his colleague Herman Afzelius, he has introduced structured, disciplined approaches to exploratory testing into two companies (and counting), and he’s been successful, despite some occasional middle-management pushback. When he’s managing and training testers, he focuses on minds before processes and tools. That’s before, not instead of: at one of the companies, he commissioned a reporting tool very much like Shmuel Gershon’s Rapid Reporter. A couple of years back, he showed me a wonderful little trick: instead of using hyphens or dots as the bullet points in your session notes, start the paragraph with a smiley emoticon for good news, and a frowny emoticon for bad news. Readers can scan the bullets to get a feel of what the tester is reporting. Instant, easy visualization! A blink test for reporting!

Kristoffer Nordström (@kristoffer_nord on Twitter) joined Petter and me in a couple of conversations. Kristoffer was one of the team leads working with Petter at UIQ Technologies when I visited there in 2008. In 2009, Petter, Herman, Kristoffer and I had a memorable Mongolian meal and a grand chat just across from Stockholm Central station, in which we described exploratory approaches, Vanguard-style values, and management resistance. Alas, we missed a chance for dinner this year, but Petter, Kristoffer, and I did compare notes on our Weltschmerz with respect to Traditionalist approaches and Traditionalist presentations in the EuroSTAR program, of which I suspect that they attended more than I did. I don’t think any of Petter, Kristoffer, or Herman have a blog. I wish they did, and if they do, I wish they’d tell me about it. Meanwhile, Kristoffer has started micro-blogging at least.

It was a pleasure to meet, finally Ola Hyltén (@ola_hylten). He attended my Tuesday morning tutorial on Test Framing, and contributed a number of valuable insights there.  (Update to this post:  I’ve just discovered, to my delight, that he’s got a blog here.  And in the most recent post, he’s writing on a topic that is near and dear to me:  parallels between testing and music.)

John Stevenson (@steveo1967) was also a keen contributor at the tutorial. John is a dedicated student of exploratory testing and systems thinking, and he writes a blog in he which stretches thinking about testing outside of the craft, which I argue is essential to advancing it. As examples of his wide-ranging perspective, look at his two posts inspired by EuroSTAR: The Human Element and Sorting the Chaff from the Wheat.

Rob Lambert (@Rob_Lambert) is one of the central figures in the Software Testing Club, an online community that provides some of the more articulate discussions on testing these days. That effort has spilled over into The Testing Planet, a periodical testing newspaper of the physical kind (remember newspapers?  News, printed on paper?). Rob is a very conscientious fellow, sharp at spotting flaws but also ready to see the good and the salvageable in the things that he observes. I admire that.

Anko Tijman (@agiletesternl) is a passionate advocate for agile approaches, and maintains a strong focus on the first phrase in the Agile Manifesto: individuals and interactions. He also frequently advocates something that I don’t think is always prominent in the Agile community: an emphasis on diversity in testing. He’s written a book that is as yet, only available in Nederlandese (Dutch), and he has a blog that he diligently updates, well, not very often at all. So the key, apparently, is to find him at a conference, see one of his presentations and chat with him, or to follow him on Twitter.

There are still at least two more EuroSTAR missives to go. More later!

EuroSTAR Trip Report, Part 2

Thursday, December 9th, 2010

In this post, I’ll highlight a few more of the people that I met at EuroSTAR 2010. Please note that because there were so many people that I’d like to mention, there’s still more to come in subsequent posts. Also, I’ve included tons of links to these people and their work. Please use those links!

Shmuel Gershon (@sgershon on Twitter) was in the Test Lab a lot, only one of the reasons that he won the Test Lab Rats’ informal yet prestigious “Most Enthusiastic Tester” award and T-shirt. Rapid Reporter, Shmuel’s tool for taking notes in exploratory testing sessions, was prominent too. Shmuel used a number of strategies for meeting other people at the conference; he announced a pizza-and-drinks session for old and new members of the Vanguard community (also known as the Rebel Alliance or, for this occasion, Danish Alliance), and he used a cute strategy for introducing himself. On a personal note, Shmuel also helped me enormously by agreeing on the spur of the moment to act as my interviewer for an upcoming EuroSTAR “Take 10” video spot. After the conference, Shmuel and I spent a pleasant afternoon at Copenhagen’s Experimentarium, browsing the exhibits, chatting, discovering bugs in the displays, and exploring patterns of exploratory testing. I was also pleased to beat him in a virtual bicycle race. Fortunately for me, my bike was the one with the seat. And the doctors say I’m recovering well from the experience.

Teemu Vesela (@teemuvesela on Twitter) received the second award from the Lab Rats: “Most Evil Tester”. He established his reputation by asking for—and getting—the Lab’s server and router passwords from the Lab Rats. His claim was that he needed that information to see if he could perform exploit of the applications that were installed in the Test Lab. But maybe, just maybe, he was testing to see if he could obtain the trust of the network administrators, just as one would try to do in a real security penetration test. Teemu exuberantly investigated several potential vulnerabilities, found some cool bugs, and enthusiastically told concise little stories about weaknesses in system defenses. And now I’ve got someone new to talk to when I want to learn quickly about potential security risks.

Henrik Andersson (Twitter: henkeandresson) is a long-time student and advocate of the practice of exploratory approaches to testing, especially within the Swedish testing community. His success has been all the more remarkable considering that, for many years, he worked for an organization that advocates strongly scripted approaches. Henrik gave an excellent talk on his experience introducing exploratory testing extremely rapidly at a large corporation that was, in general, resistant to the idea. His focus was on the role of champions—passionate people who will support and sustain excellent work, philosophically much like those in the Vanguard. Henrik described his approach: little experiments followed by intensive debriefs; granting people the freedom and responsibility to design and evaluate their work; emphasizing the roles of discovery, learning, and feelings. Within the constraints, he was quite successful, but once again incomprehending middle management provided only tepid support.  Thanks to the ubiquitous Markus Gärtner (of whom more quite soon), here’s a detailed account of Henrik’s presentation.

Fredrik Rydberg—someone whom I didn’t know before and (alas!) did not meet in person—gave a superb experience report titled “Can Exploratory Testing Save Lives?” on using exploratory approaches in a regulated, medical context.  His conclusion was an emphatic Yes.  There’s a lot of nonsense in our craft that suggests that you can’t or shouldn’t do exploratory testing in a mission- or safety-critical environment. In fact, as Fredrik made clear, it’s exactly the opposite: if you want to reduce risk and save lives, you must take an exploratory approach to develop tests, to incorporate new information, to continuously re-evaluate your work, and to reveal previously unrecognized risks. Fredrik aptly pointed out that curiosity, patient communication, and networking skill are crucial to a successful exploratory approach; indeed, they’re important to collaborative work of any kind. I hope to meet Fredrik and chat with him more in the future. We need more stories from him, and more stories like his.

Carsten Feilberg (@carsten_f on Twitter) blew me away at the CAST 2008 conference, where he provided a mischievous foreign element during a simulation in Jerry Weinberg’s Tester’s Communication Clinic. His impishness appealed to me, but there’s far more to Carsten than that. At EuroSTAR 2010, he gave a fabulous talk on Session Based Test Management (SBTM). One of the biggest takeaways was the simplest, yet psychologically the most powerful: he took the subtle step of renaming the practice to “Managing Testing Based on Sessions” (MTBS), in order to emphasize to managers the significance of the management aspect. This allowed him to obtain rapid buy-in from skeptical managers at his organization. That simple trick reminded me of Thomas Huxley’s wonderful observation on Charles Darwin’s On the Origin of Species: “How stupid of us all not to have thought of that.” He also provided an elegant visual metaphor for the development process. He started by showing a picture of a cartoon elephant (“the requirements”)—smooth, uniform, clear lines.  Then, over a part of the cartoon elephant, he superimposed the kind of view we’d see after testing: a photograph of the same part of a real elephant—wrinkled, lumpy, hairy. It was a great image, and a great visual explanation. He gradually revealed the bits and pieces of the elephant—and noted that the real elephant had tusks, where the cartoon elephant had none. Exploring the actual product allows us to see things that we wouldn’t see otherwise.

Carsten’s experience report underscored the fact that SBTM/MTBS makes exploratory testing more legible—more readable, more understandable—for managers who might otherwise see it as undisciplined, unstructured, or incomprehensible. I’ve written a couple of blog posts on some approaches that might help clear things up here, here, and here particularly.  Yet if you want advice on how to persuade management to recognize and adopt exploratory testing in your organization, it would also be a really good idea to contact Carsten.  Alas, his presentation slides are not yet online, but Markus Gärtner’s report on Carsten’s talk is.

Ah, Markus Gärtner, another of those fellows who was everywhere, all the time, and he has the blog posts to prove it. In the Test Lab, he was a vigourous participant, asking questions, probing for ideas, and sharing insights. At the conference presentations, Markus was like an old-fashioned on-the-scene radio reporter, “blogcasting” live and typically posting his transcription of the presentation a few seconds into the question-and-answer period. He also gave a presentation on self-education for testers, which for the Vanguard means not only study, but actively practicing testing. Apropos of that, Markus was one of the founders of the European chapter of the Weekend Testing. And apropos of that

Weekend Testing was started in Bangalore, India, by Parimala Shankaraiah (@curioustester on Twitter), Manoj Nair (@manoj_mv), Sharath Byregowda (@sharathb on Twitter)), and Ajay Balamurugadas (@ajay184f on Twitter). The latter was at EuroSTAR to tell the story of how the movement began and how it has developed since then. Inspired by Pradeep Soundararajan (@testertested on Twitter), and soon assisted by Santosh Tuppad (@santhoshst on Twitter), the founders decided to take responsibility for their own education and training. On August 15, 2009, they began meeting online on Saturday afternoons to practice testing, to challenge each other, and to help each other develop skills. Sessions were structured as an hour of testing (typically in pairs) and an hour of group discussion and sharing afterwards. Side effects quickly followed: their reputations blossomed; several open source projects benefitted from their testing; and the larger community became engaged. Weekend Testing quickly sprouted chapters in Mumbai, Chennai, Europe, Australia/New Zealand, and (finally!) North America. Ambitious and eager testers have come out of the woodwork, and more senior colleagues have facilitated sessions. The great conversation of skilled testing goes on, and the Vanguard is growing! I’ll mention more of its people in the next post.

EuroSTAR Trip Report, Part 1

Tuesday, December 7th, 2010

Way way back in 2003, Bret Pettichord first published a paper on schools of software testing. The paper was controversial. Some people found it helpful to identify different schools of thought, for the purpose of understanding ways in which reasonable people might disagree reasonably.  Others found even the mention of disagreements within the field to be distasteful and divisive.  Some people identified with particular schools. Others, sometimes indignantly, refused to be pigeonholed. Yet it’s clear that in any field of endeavour, including testing, there are always communities of thought and practice. Sometimes those communities are isolated; sometimes there are trading zones between them.

No matter how one might label the communities, two broad categories were apparent to me at this year’s EuroSTAR conference. One group seems to focus on testing in terms of confirmation, verification, validation, quality assurance; getting the right answers to prescribed questions; checking. This group’s approach includes a strong focus on artifacts—requirement documents, detailed test plans, and scripted test cases. This group (let’s call it the Traditionalists) also seems to focus on processes and tools, on negotiated contracts, and on following plans—items on the right side of the Agile Manifesto. I don’t claim membership in the Agile School. Although though I greatly admire the principles in the Manifesto, for me, the first thing to look at is the project’s context, and to proceed accordingly. The Traditionalistas, as I see it, emphasize the Agile Manifesto’s “things on the right”. Probably they do so with the desire to dispel variability, subjectivity, and unpredictability from testing.  I try to be empathetic towards those who advocate the things on the right, since those aren’t unreasonable things to want; it’s just unreasonable, in my view, to believe they’re the more important things in the complex, messy, human, and constantly changing world of software development.

The other, significantly smaller—and, in general, younger—group that I observed at EuroSTAR sees testing as questioning, exploration, discovery, investigation, and learning—and quality assistance. Let’s call that group the Vanguard. The Vanguard realizes that getting the right answers is important, but asking the right questions is more important—and recognizing that today’s “right questions” today are probably different from yesterday’s “right questions” is more important still. In broad strokes, the Vanguard prefers

experience reports over “best practice” talks
conversation over lectures
hands-on exercises over PowerPoint presentations
tools for investigation over tools for confirmation
dialogue over monologue
sitting in a circle over classroom format
finding things out over hearing the answer

And, as in the Agile Manifesto, they recognize value in the things on the right, but they value the things on the left more.

The Vanguardistas are eager to participate in testing exercises, and to exchange testing skills by example and by dialog. The Vanguard raises some difficulties for traditional trainers and presenters, because the Vanguard tends to want to ask questions and challenge authority—and as a trainer and presenter, I think that’s great. Many of the Vanguardistas participate in or organize Weekend Testing sessions. Almost all of them are on Twitter. They want to revive and reinvent testing as a sophisticated art that requires vigourous critical thinking. They’re indefatigably curious and engaged, and they’re becoming recognized as leaders in their community and in the testing craft.

One hallmark of the Vanguard at EuroSTAR was that they gravitated towards doing testing in the Test Lab, once again run by James Lyndsay (@workroomprds on Twitter) and Bart Knaack (@Btknaack on Twitter) after their impressive success at EuroSTAR last year. This year, 180 people visited the Test Lab. Though probably a minority, that’s a significant percentage of the overall attendees, and is all the more remarkable because, for space reasons, the Test Lab was quite a distance away from most of the presentations. This year there were more applications to test, more sharply focused vendor presentations, specific guidance for those who needed it, and lots of pairing and sharing. For me, one of the more memorable events was a relatively impromptu exploratory testing management roundtable, facilitated by James, with more than 20 people attending—remarkable because the event wasn’t noted specifically as a scheduled part of the conference programme; it was set up in the Test Lab, advertised by word of mouth, and fundamentally collaborative. The roundtable was one of those things that put the confer back in conference.

Of many high points of the roundtable conversation, the big one for me was the group’s recognition that testers don’t need to be domain experts from the outset of a testing assignment. Instead, testers can partner with domain experts in review and hands-on testing sessions, and in that collaboration get some excellent testing work done immediately. An exploratory testing cycle—test design, test execution, test result interpretation, learning, debriefing—drives rapid and highly effective learning about the domain. As Rob Sabourin (more on Rob later) articulated it: “Here’s a beautiful charter for a test session: Sit with a customer/user and ask ‘What gets in the way of you doing your work?'”

James and Bart were assisted this year by the Test Lab apprentices, Henrik Emilsson (@henrikemilsson on Twitter) and Martin Jansson (@martin_jansson on Twitter). At EuroSTAR 2011, management of the Test Lab will pass to Henrik and Martin. It’s in good hands. Henrik and Martin are members of a blogging cabal called thoughts from the test eye, which has been producing incisive, thoughtful reflections on testing since February 2008. An outstanding example is a blog post announcing their own list of software quality characteristics, in which they build on one of the pillars of James Bach‘s Heuristic Test Strategy Model. But that’s just one example. Read the back issues and put the new ones in your feed reader.

Another member of the test eye collaborative is Rikard Edgren. Rikard was one of the conference chairs of EuroSTAR this year. He seems to have found a way to violate some fundamental law of physics by being everywhere at the same time; whenever I turned around, he was there with an expression on his face that reflected his keen observational skill and his sly humour. I’ve been lucky to have many interesting chats with him, not only this year but in years previous.

More on EuroSTAR 2010 tomorrow.