Blog Posts for the ‘Community’ Category

Rising Against the Rent-Seekers

Monday, August 25th, 2014

At CAST 2014, a quiet, modest, thoughtful, and very experienced man named James Christie gave a talk called “Standards: Promoting Quality or Restricting Competition?”. The talk followed on from his tutorial at EuroSTAR 2013 on working with auditors—James is a former auditor himself—and from his blogs on software standards over the years.

James’ talk introduced to our community the term rent-seeking. Rent-seeking is the act of using political means—the exercise of power—to obtain wealth without creating wealth; see and One form of rent-seeking is using regulations or standards in order to create or manipulate a market for consulting, training, and certification.

James’ CAST presentation galvanized several people in attendance to respond to ISO Standard 29119, the most recent rent-seeking scheme by a very persistent group of certificationists and standards promoters. Since the ISO standard on standards requires—at least in theory—consensus from industry experts, some people proposed a petition to demonstrate opposition and the absence of consensus amongst skilled testers. I have signed this petition, and I urge you to read it, and, if you agree, to sign it too.

Subsequently, a publication named Professional Tester published—under an anonymous byline—a post about the petition, with the provocative title “Book burners threaten (old) new testing standard”. Presumably such (literally) inflammatory language was meant as clickbait. Ordinarily such things would do little to foster thoughtful discussion about the issues, but it prompted some quite thoughtful reactions. Here’s one example; here’s another. Meanwhile, if the author wishes to characterize me as a book burner, here are (selected) contents of my library relevant to software testing. Even the lamest testing books (and some are mighty lame) have yet to be incinerated.

In the body text, the anonymous author mischaracterises the petition and its proponents, of which I am one. “Their objection,” (s)he says, “is that not everyone will agree with what the standard says: on that criterion nothing would ever be published.” I might not agree with what the standard says, but that’s mostly a side issue for the purposes of this post. I disagree with what the authors of the standard attempt to do with it.

1) To prescribe expensive, time-consuming, and wasteful focus on bloated process models and excessive documentation. My concern here is that organizations and institutions will engage in goal displacement: expending money, time and resources on demonstrating compliance with the standard, rather than on actually testing their products and services. Any kind of work presents opportunity cost; when you’re doing something, most of the time it prevents you from doing something else. Every minute that a tester spends on wasteful documentation is a minute that the tester cannot fulfill the overarching mission of testing: learning about the product, with an emphasis on discovering important problems that threaten value or safety, so that our clients can make informed decisions about problems and risks.

I am not objecting here to documentation, as the calumny from Professional Tester suggests. I am objecting to excessive and wasteful documentation. Ironically, the standard itself provides an example: the current version of ISO 29119-1 runs to 64 pages; 29119-2 has 68 pages; and 29119-3 has 138 pages. If those pages follow the pattern of earlier drafts, or of most other ISO documents, you have a long, pointless, and sleep-inducing read ahead of you. Want a summary model of the testing process? Try this example of what the rent-seekers propose as their model of of testing work. Note the model’s similarity to that of a (overly complex and poorly architected) computer program.

2) To set up an unnecessary market for training, certification, and consultancy in interpreting and applying the standard. The primary tactic here is to instill the fear of being de-certified. We’ve been here before, as shown in this post from Tom DeMarco (date uncertain, but it seems to have been written prior to 2000).

Rent-seeking is of the essence, and we’ve been here before in another sense: this was one of the key goals of the promulgators of the ISEB and ISTQB. In the image, they’ve saved the best for last.

The well-informed reader will note that the list of organizations behind those schemes and the members of the ISO 29119 international working group look strikingly similar.

If the working group happens to produce a massive and opaque set of documents, and you’re in an environment that claims conformance to the 29119 standards, and you want to get some actual testing work done, you’ll probably find it helpful to hire a consultant to help you understand them, or to help defend you from charges that you were not following the standard. Maybe you’ll want training and certification in interpreting the standard—services that the authors’ consultancies are primed to offer, with extra credibility because they wrote the standards! Good thing there are no ethical dilemmas around all of this.

3) To use the ISO’s standards development process to help suppress dissent. If you want to be on the international working group, it’s a commitment to six days of non-revenue work, somewhere in the world, twice a year. The ISO/IEC does not pay for travel expenses. Where have international working group meetings been held? According to the Web site, meetings seem to have been held in Seoul, South Korea (2008); Hyderabad, India (2009); Niigata, Japan (2010); Mumbai, India (2011); Seoul, South Korea (2012); Wellington New Zealand (2013). Ask yourself these questions:

  • How many independent testers or testing consultants from Europe or North America have that kind of travel budget?

  • What kinds of consultants might be more likely to obtain funding for this kind of travel?

  • Who benefits from the creation of a standard whose opacity demands a consultant to interpret or to certify?

Meanwhile, if you join one of the local working groups, there are two ways that the group arrives at consensus.

  • By reaching broad agreement on the content. (Consensus, by the way, does not mean unanimity—that everyone agrees with the the content. It would be closer to say that in a consensus-based decision-making process, everyone agrees that they can live with the content.) But, if you can’t get to that, there’s another strategy.

  • By attrition. If your interest is in promulgating an unwieldy and opaque standard, there will probably be objectors. When there are, wait them out until they get frustrated enough to leave the decision-making process. Alan Richardson describes his experience with ISEB in this way.

In light of that, ask yourself these questions:

  • How many independent consultants have the time and energy to attend local working groups, often during otherwise billable hours?

  • What kinds of consultants might be more likely to support attendance at local working groups?

  • Who benefits from the creation of a standard that needs a consultant to interpret or to certify?

4) To undermine the role of skill in testing, and the reputations of people who discuss and promote it. “The real reason the book burners want to suppress it is that they don’t want there to be any standards at all,” says the polemicist from Professional Tester. I do want there to be standards for widgets and for communication protocols, but not for complex, cognitive, context-sensitive intellectual work. There should be standards for designed things that are intended to work together, but I’m not at all sure there should be mandated standards for how to do design. S/he goes on: “Effective, generic, documented systematic testing processes and methods impact their ability to depict testing as a mystic art and themselves as its gurus.” Far from treating testing as a mystic art, appealing to things like “intuition” and “experienced-based techniques”, my community has been trying to get to the heart of testing skills, flexible and responsive coverage reporting, tacit and explict knowledge, and the premises of the way we do testing. I’ve seen no such effort to dig deeper into these subjects—and to demystify them—from the rent-seekers.

Unlike the anonymous author at Professional Tester, I am willing to stand behind my work, my opinions, and my reputation by signing my name and encouraging comments. Feel free.

—Michael B.

Exegesis Saves (Part 3) Beyond the Bromides

Sunday, January 23rd, 2011

Over the last few blog posts, some colleagues and I have been analyzing this sentence:

“In successful agile development teams, every team member takes responsibility for quality.”

Now, in one sense, it’s unfair for me to pick on this sentence, because I’ve taken it out of context. It’s not unique, though; a quick search on Google reveals lots of similar sentences:

“Agile teams work in a more collaborative and open manner which reduces the need for documentation-heavy, bureaucratic approaches. The good news is that they have a greater focus on quality-oriented, disciplined, and value-adding techniques than traditionalists do. However, the challenge is that the separation of work isn’t there any more—everyone on the team is responsible for quality, not just ‘quality practitioners’. This requires you to be willing to work closely with other IT professionals, to transfer your skills and knowledge to them and to pick up new skills and knowledge from them.”

“In Agile software development, the whole team is responsible for quality, but there are many barriers to accomplishing that goal.”

“So what does testing now need to know and do to work effectively within a team to deliver a system using an agile method? The concept of ‘the team being responsible for quality’ i.e. ‘the whole team concept’ and not just the testing team, is a key value of agile methods. Agile methods need the development team writing Unit tests and/or following Test First Design (TDD) practices (don’t confuse TDD as a test activity as in fact it is a mechanism to help with designing the code). The goal here is to get as much feedback on code and build quality as early as possible.”

“As we have seen quality is the responsibility of every team member in an agile team, not just the developers. Every team member has a role to play in building quality in.”

“The responsibility for quality was shifted to the whole team. Each of the different roles is responsible for doing some form of testing. Programmers, architects, analysts, and even managers are all intimately involved with testing activities and work closely together to achieve quality goals.”

“In traditional systems, the responsibility for quality is mainly delegated to testing teams that must make sure the code is of high quality. Agile thinking makes quality a collective responsibility of the customer, the developers and the testers all the time from the first, to the last minute of a project. The customer is involved in quality by defining acceptance tests. The developers are involved in quality by helping the customers write the tests, by writing unit tests for all the production code they write and the testers are involved by helping the developers automate acceptance (customer) tests and by extending the suite of automated tests.”

“I’m an advocate of any methodology that empowers engineers to work toward higher standards of software integrity. Agile and test-driven methodologies have found a way to do that by distributing ownership and responsibility for quality. And with so many ways to make static analysis boost the type of automated testing this requires, agile is a better formula for better code that can keep up with shorter scrum cycles and produce frequently “potentially shippable” products.”

“This then leads back to my original question. Who is responsible for quality? The answer is everyone.”

“The corollary to this rule is that testers cannot be responsible for quality; developers must be. The Agile methods put the responsibility for quality precisely where it belongs, with the developers.”

“In a successful team, every member feels responsible for the quality of the product. Responsibility for quality cannot be delegated from one team member to another team member or function. Similarly, every team member must be a customer advocate, considering the eventual usability of the product throughout its development cycle. Quality has to be built into the plans and schedules. Use bug allotments, iterations devoted to fixing bugs, to bring down bug debt. This will reduce velocity which may provide enough slack in future iterations to reduce bug rates.”

So the sentence that I quoted is by no means unique.“>Here’s the source for it. It’s an article by Lisa Crispin and Janet Gregory. You can read the rest of the article, and make your own decisions about whether the rest of it is helpful. You may decide (and I’d agree with you) that there are several worthy points in the article. You may decide that I’ve been unfair (and I’d agree with you) in excerpting the sentence without the rest of the article, especially considering that until now I’ve ignored the sentence that follows immediately, “This means that although testers bring special expertise and a unique viewpoint to the team, they are not the only ones responsible for testing or for the quality of the product.”

The latter sentence certainly clarifies the former. I would argue (and this is why I brought the whole matter up) that, in context, the second sentence renders the first unnecessary and even a distraction. Please note: my intention was not to critique the article, nor to launch a personal attack on Lisa and Janet. As I’ve said, I have no doubt that Lisa and Janet mean well, and their article contains several worthy points. My goal instead was to test this often-uttered tenet of agile lore. That’s why I didn’t reveal the source of the sentence initially; the article itself is not at issue here.

Yet the sentence, found in so many similar forms, is a bromide. My Oxford Dictionary of English says that a bromide is “a trite statement intended to soothe or placate”. As philosophers might say, the sentence doesn’t do any real explanatory work; it doesn’t make any useful distinctions. Even in context, it’s often unclear as to whether the sentence is intended be descriptive (“this is the way things are”) or normative (“this is the way things should be”).

To highlight my greatest misgivings about the slogan, let’s restate it, replacing the word “quality” with Jerry Weinberg’s definition of it:

“In successful agile development teams, every team member takes responsibility for value to some person(s).”

Every time I see the whole-team-responsible-for-quality trope, I find myself wondering responsibility to whom, quality according to whom, and what it means to take responsibility. Figuring out what we intend to achieve, and how we are to achieve it, are among the harder problems of software development. (One can see this being played out in the Craftsmanship Skirmishes that are going on as I write.)

Here’s what would make the conversation more meaningful to me. I suggest that we who develop and write about software…

  • Consider quality not as something simple, objective, and abstract, but as something messy, subjective and very human. Quality isn’t a thing, but rather a complex set of relationships between products, people, and systems. As such, quality shouldn’t be swept under the rug of a vague slogan.
  • Remember that there are as many ways to think about quality as there are people to think about them, and that in a software development project, people’s interests will clash from time to time. Those conflicts will require some people to concede certain values in favour of other values. As Jerry Weinberg has pointed out, decisions about quality are political and emotional. Sorting out those conflicts will require us to identify who has the power to make the decision, and to contribute to empowering that person when appropriate. That in turn will require us to develop skill, courage, and confidence, tempered with patience and humility.
  • Apropos of the last point, we must keep in mind that teams are collections—or systems—of individuals. it’s a nice idea to think that the whole team makes decisions collectively, but in reality some person—the product owner—must be granted the ultimate authority to make a decision. If the team is to be successful, there’s an implicit contract: the product owner must enable, trust and empower the members of the team to design and perform their work collaboratively, in the best ways that they know how; and the members of the team must inform, trust and empower the product owner, such that she can make the best decisions possible.
  • Since quality means “value to some person(s)”, be specific about what particular dimension of value we mean, and to whom. For example, in a particular context, if we want “quality” to mean “bug prevention,” let’s say precisely that. Then let’s recognize the ways in which certain approaches towards preventing bugs might represent a threat to someone’s current interests or to their personal safety rules. If, in a particular context, we want quality to mean “problems solved for customers”, let’s say precisely that. Then let’s recognize that there are many approaches to solving problems, and that some problems might be solved by writing less software, not more. If we want “quality” to mean “many features in a product”, let’s say precisely that. Then let’s recognize how “many features” can satisfy some people—while adding complexity, development time, and expense to a product, thereby confusing and annoying other people. In other words, let’s use the word “quality” in a more careful way, which starts with deciding whether we want to use the word at all.
  • Since quality is a complex notion, we must learn not only to declare quality in terms of things that we expect and prescribe. Those things are important, to be sure. Yet we must also prepare to seek the unexpected, to make discoveries, and to respond to change.
  • Rather than striving for influence (which to me has echoes of the quality assurance or quality control mindset), testers in particular must strive to develop understanding and awareness of the system that we’re being asked to test. To me, that’s our principal product: learning about the system and reporting what we’ve learned to help the project community to gain greater understanding of the system, the problems, and the risks.
  • Consider responsibility not in terms of something people take, but as something that people accept, and also as something that people grant, share, and extend to one another. To me, that means contributing to an environment in which everyone is empowered to create and defend the value of each other’s work. That includes offering, asking for and accepting help—and dealing with unpleasant news appropriately, whether we’re delivering it or receiving it.

There have been several pleasures in working through this exercise—most notably the transpection with my colleagues on Twitter and in Skype. But there’s been another: rediscovering and re-reading the Declaration of Interdependence and George Orwell’s Politics and the English Language.

Note that I’m saying that these ideas would make the conversation more meaningful to me. You may have other thoughts, and I’d like to hear them, so I hope you’re willing to share them.

Exegesis Saves! (Part 2) Transpection with James Bach

Friday, January 21st, 2011

Last evening, after a long session of collecting and organizing a large number of contributed responses to yesterday’s testing challenge, I was going over my own perspectives on the sentence

“In successful agile development teams, every team member takes responsibility for quality.”

James Bach appeared on Skype, and we began an impromptu transpection session. It went more or less like this:

James: I saw your original challenge and a couple of replies. Are you familiar with the “tragedy of the commons”? That’s what the sentence reminds me of. However, I bet there is something real behind that statement; something good. I have been on management teams where I felt we were all responsible for quality, but not for all quality equally. We each had our jobs to do.

Anyway, I think “tedious” is a good descriptor. The sentence is a bland truism, like “honesty is good”. You can re-render it like this: “On a successful agile project, you won’t find anyone irresponsible about quality.” Oh THANKS for that insight.

Michael: I’ve got a bunch of problems with it. Is it a descriptive definition or normative definition? What is being distinguished here? The successful vs. the unsuccessful? The agile vs. the non-agile? Every team member vs. most team members vs. no team member? The responsible vs. the irresponsible? Does quality here mean critical thinking, or writing acceptance tests, or the absence of bugs, or making the customers happy? What’s the scope for quality here? Could you ever get anyone to admit seriously that they’re not responsible for the quality of their own work? It seems to me that, with those things all in play, the sentence doesn’t do any useful work. “On a successful agile team, every team member is responsible for apple pie.”

James: You could contrast it with waterfall: On a successful waterfall project, only some people need to be responsible for quality. That makes waterfall seem better than agile, acutally; less risk.

Michael: For somebody!

James: Or you could contrast it with the default: Under normal circumstances, people don’t take responsibility for quality (really?), unlike successful agile projects, where everyone does. Or you could take it to logical conclusion: if two people on an agile project disagree about quality, they must work through the diagreement or else one of them must leave the project (really?). Which means: Successful agile projects consist only of people who are in complete agreement about quality.

What does “responsibility for quality” actually mean in that context? Does it mean “making the product better?” Or does it mean “accepting how the product is?” Maybe the intention is: developers have to fix any bug that anyone on the project wants them to fix.

Michael: Yes. All kinds of questions are being begged, it seems to me. Now, to be fair, I haven’t yet provided a link to the article in which this exact sentence appeared (I’ll talk about that in my own reply), but when I look up keywords from the sentence on Google, I find a ton of articles that beg the quality and responsibility questions. It seems to me that the intention is usually this: “On an Agile team, everyone is responsible for preventing bugs.” That’s a nice idea, but even that statement would have serious problems. Or maybe the intention is “On an Agile team, testers get to go to the meetings too.” To some people, it means “on an Agile team, the programmers actually test their own code.” In fact, I remember in the early years of the Agile movement, a few programmers would tell me that they’d never worked with testers who had delivered any real value to the project. When I considered some of the places I had visited and testers I had met, I was dismayed to find that claim credible. In such contexts, really conscientious programmers would tend to be extra diligent about testing their own work, and would tend to assert that they and they alone were responsible for its quality. That’s my understanding of part of the genesis of XP—programmers asserting that with respect to coding bugs at least, the buck stopped with them. That seems like it could be a positive subtext.

James: I think I see this subtext, too: “On an agile project, there is no social structure or order. We are a perfect Einstein-Bose condensate of spontaneous harmony. Everyone does everything. Communication is instant. Agreement is assured. All skills are held in common. All experiences are equivalent.”

Which in practice means: The Strong Rule The Weak.

Michael: Ouch.

James: The sub-sub text is then this: “Don’t ask about social order. We don’t like to talk about that.”

Michael: “…but we do have this sentence that you’re not really supposed to question. It’s a form of Word Magic, so please don’t mess with it.”

James: How about this, then: “On an agile project, one thing that is beyond question is that everyone is resposible for quality. Also, unicorns are pretty.”

Still more to come. Stay tuned.

Exegesis Saves! (Part 1)

Friday, January 21st, 2011

This morning, I read a sentence that bugged me.

“In successful agile development teams, every team member takes responsibility for quality.”

I’ve seen sentences of that general form plenty of times before. Whether I’ve reacted or not, they’ve always bugged me, and today I decided to probe into why.

Rather than doing so on my own, I thought it would be more fun and more interesting to involve my community, so I posted a challenge on Twitter. If I want to get any other work done, I’m going to have to learn to stop doing that. I posted:

“Thinking Thursday. Test this sentence: ‘In successful agile development teams, every team member takes responsibility for quality.'”

Although I have great respect for my colleagues, I hadn’t anticipated so many interesting replies. So, today, my summary of the responses on Twitter. Soon, a brief conversation between James Bach and me, and shortly thereafter, my own assessment.

Questioning the Mission

Adam Goucher was the first to reply. He responded to my tweet by noting, “I could also check it by putting into a wordprocessor and seeing what is underlined.” I read this, laughed, and replied, “You could. But I ask for testing. :)” Adam promptly replied that he had thus clarified the mission. Yes; absolutely. One tactic for refining your mission is to make an offer. Acceptance, rejection, or other reactions to the offer may be revealing.

Before testing the sentence, Shrini Kulkarni immediately (and wisely) questioned the testing assignment. “First question: Why are you asking this question, and how will you use my answer? Question #2: What is your motivation for asking this question and how will you evaluate my response?”
Those were splendid questions. I emphasize, especially for all those new to testing: if you feel uncertain or confused, it’s vitally important to question the mission to reduce the risk that your assumptions don’t align with your client’s assumptions.

I answered. “I’m exploring what bugs me (and others) about the sentence. I’m not sure how (or even if) I’m going to use the information just yet; that’s part of my exploration.” That’s the why, my motivation. In answer to the evaluation question, I listed intellectual or emotional stimulation; insight, epiphanies, and reminders of old epiphanies as factors. I didn’t add then (but I will now) that I didn’t think of it as a competition. (To that end, I’ve presented as many of the replies as I’ve been able, and although I wasn’t intending to evaluate them in a competitive way, I can’t help but be impressed.)

Shrini had a couple more questions. “What is your interpretation of ‘testing’ of a sentence like this?” The goal, he said, was to know how my notion of testing the sentence might be different from his. At the time, I was away from my computer, so I didn’t respond to Shrini right away. So Shrini did something else that an expert tester does; if you can’t get an answer, state your assumptions. “One way I can think of ‘testing’ a sentence is to check if it is true. So my question would be “how do you [missing word] it is true?” (I believe the missing word was a typo.) “Another interpretation of testing a sentence—a linguistic construct—is to check if it is formed as per the rules of grammar.” So here Shrini, in the face of an ambiguous assignment, gave two possible interpretations and got to work, focusing on the one that he figured was the deeper problem.

Martin Jansson also asked for clarification: “When asking me to test this, are you the only stakeholder?” I was away when that message came in too. In reply, I’ll say now that I was the only client, but in a way, lots of people might be stakeholders. I leave it as an exercise to figure out who those stakeholders might be.

Questioning the Context of the Sentence

In addition to questioning the mission, questioning the context is crucial. Martin Jansson had a bunch of context questions. “Since Twitter limits only to 140 chars, can I see the full length of what was originally intended? Is this sentence localized and is originally from another language? Can I see the original one? What other sentences are connected to the one we are testing? What will the system and its environment look like?” Griffin Jones asked, “What is the history of this sentence in this context? Is this sentence the image we want to project as a company or group? Do our peers/competition make a similar claim? Do we publicly claim to follow that sentence?”

Identifying the Problems

The very first reply that I received was from Florin Duriou, who responded simply and directly, “too generic”. As Lynn McKee said, “Many words in that sentence are subjective.” I agree with that, so let’s get specific. How?

Shrini offered two approaches. “I do little of Jerry Weinberg’s “Mary had a little lamb” exercise to see possible meanings of every word and their combinations.” That exercise, from Exploring Requirements, involves going through the simple sentence “Mary had a little lamb,” and emphasizing each word in turn to see what other interpretations might be lurking.

  • “MARY had a little lamb” (but Joe didn’t, so he was jealous)
  • “Mary HAD a little lamb” (but she doesn’t any more)
  • “Mary had A little lamb” (then she had two—and now she has a flock)
  • “Mary had a LITTLE lamb” (but too much lamb would have been bad for her diet)
  • “Mary had a little LAMB” (why did you think I said “ham”?)

Shrini’s second approach: “Testing a sentence, I look for ‘words’ in it—like successful, agile, development, teams, team, member, quality, responsibility.” By putting “words” in quotes, I think Shrini was emphasizing that every one of these ideas are concepts, constructs, thought-stuff. So let’s look at some of those words and constructs in more detail.


Success is context-dependent. Griffin, Lynn, Peter Walen, and Stephen Hill all questioned the meaning of “success”. Simon Morley wanted to know whether we might deem a team successful without knowing what the team was doing or producing-and whether its efforts were desired. Both Pete Houghton and Martin asked whether criteria for success included shipping on time. Martin also asked if quality was the only factor that makes an Agile team “successful”. “What if they hate each other and learn nothing from what they do?”

“Responsibility”, “Agile”, “Team”

What does “responsibility” mean? Áine McGovern held that programmers should take responsibility for testing at low levels before testers get their hands on the product; that everyone is are responsible for a product that works well. Yet responsibility might mean “blame”, as both Peter Houghton and Peter Walen observed. What does “agile” mean? As Martin joked, “Did we mean Agile or perhaps AGILE? ‘Cause being AGILE is so much better than just lowercase agile.” Does the Agile part even matter? Anand or Komal Ramdeo ( I don’t know which; they have a joint Twitter account) noted that “agile” could be removed from the sentence without loss of meaning, if we believe that teams are successful when every team member takes responsibility. Pete Houghton thought to question what we mean by “team”. Griffin even questioned the role of “in” in the sentence.

Words in Combination

Joining all these words into a sentence leads to a kind of combinatorial explosion of possible interpretations. Lynn noted that success might not equal quality, depending on the project or organizational goals. I agree—and the project and organizational goals might come into conflict from time to time too, as the organization might have many projects on the go, and they projects might compete for resources.

Simon asked what “taking responsibility for quality” might involve or exclude. Peter Walen asked if we could infer that in non-agile teams, no team members are responsible for quality—or if successful waterfall teams didn’t need to worry about quality. Tim Western and had another variation: unsuccessful agile development teams don’t take responsibility for quality? Michel Kraaij asked pointedly, “So if the quality sucks, no one takes responsibility?” Adam Brown had yet another variation: wouldn’t every team member take responsibility for quality even if the project was unsuccessful?

Our sentence might be subject to the graveyard fallacy, a central theme of The Black Swan. The successful often attribute their success to some factor or practice or approach; the unsuccessful who used the same factor, practice, or approach but who were less lucky don’t survive to tell of their experience with the factor—and people seem disinclined to listen to the unsuccessful.  What can you learn from losers, after all? Nassim Nicholas Taleb, in The Black Swan, retells a story from Marcus Tullis Cicero, a tale of skeptical inquiry. “One Diagoras, a nonbeliever in the gods, was shown painted tablets bearing the portraits of some worshipers who prayed to survive the subsequent shipwreck. The implication was that praying protects you from drowning. Diagoras asked, ‘Where were the pictures of those who prayed, then drowned?’ The drowned worshippers, being dead, would have a lot of trouble advertising their experiences from the bottom of the sea. This can fool the casual observer into believing in miracles.” (You can keep reading here.) That’s what I thought of when I read Peter Houghton’s reply: “What about the unsuccessful teams where everyone also takes responsibility?”


Griffin asked for the definition of quality in this context. Markus Deibel asked if the notion of quality was based on each team member’s definition of quality, or on a common set of quality rules. Good questions; as Anand (or Komal) Ramdeo put it, “‘quality’ might have different meaning to different people—so everyone is responsible for quality but the product can still suck.” Quite right, I would argue—with the additional observation that the product might suck to some people and not to others.

Adam Goucher suggested replacing “quality” in the troublesome sentence with “the product”, as customers likely care more about that then the “quality”. I think Adam’s suggestion is to focus on something more concrete (the product), and less abstract. A good idea, but I think it shifts the problem rather than confronting it. Whatever people have to say about the product, their evaluation is an expression of a quality judgement. Martin suggested that in a successful agile development team, everyone would know that quality is abstract—and that its meaning is therefore not to be taken for granted. I agree (yet everyone? would?), and I’d be specific about this: quality isn’t an intrinsic and objective attribute of something. Instead, it’s a relationship between something and some person.

More Probing Questions

Griffin asked “compared to what?” A little while ago I jokingly suggested to Markus Gärtner that he write a macro that would at a keystroke issue the question “compared to what?” Zeger Van Hese reported that I had triggered his macro successfully. “Compared to what? Successful to whom? Quality to whom? Every time? Really?”

Zeger and Martin both raised interesting questions about equality and responsibility. Does every member of the agile team define quality equally? Can all team members ever be equally responsible for quality? Is it even possible to take responsibility for quality in the team? (For my part, I’ll speak to this issue later.) Zeger even raised what I will now call The Animal Farm Take on Responsibility: “I can’t help but thinking ‘…but some take more responsibility than others,'” he wrote.

The explicit talk of equality and implicit reference to power reminded me of Jerry Weinberg’s observation that decisions about quality are always political, made by people who have the authority—the political power—to make them.

Deciding What and How to Observe

Griffin emphasized a question that we must ask in addition to “compared to what?” He wanted to know “according to whom”. That’s important because observations, comparisons, and measurements are never value-neutral; they always depend upon at least one person, and what and how each person observes. “What would I see/hear/feel when a ‘team member’ ‘takes responsibility’?” Simon adds, “What would a team member /not/ ‘taking responsibility for quality’ look like?” Anand (or Komal) asks, “Is there any scale to measure responsibility? How do you compare more or less responsible?”

No Problem

Some people found no problem at all with the sentence. Alan Cooper retweeted it, adding “True”. Bill Clark responded to that, saying, “If the team looks good, everyone on it looks good. A bunch of lone coyotes running here and there not so much.” Ron Jeffries said, “The sentence is perfect. By the way, what did you want it to do?” An important question, but I might have placed it before my evaluation of the sentence’s perfection.

All of these responses were interesting and valuable. The most gratifying one, though, came from Zeger Van Hese, who raced to his blog before I could get to mine. You can read his response here.

More to come!

Hire Ben Simo!

Tuesday, August 31st, 2010

I have four or five blog posts in the hopper, each almost ready to go. I’m working on a whole book and a chapter of another one, and I’m on a deadline that I’m about to blow. The kids are still out of school, and I really should be cooking dinner right now. And yet…

As I write, one of the best testers that I know is looking for work. His name is Ben Simo. He lives in Colorado Springs, Colorado (my understanding is that he’s willing to relocate). He’s well-versed in LoadRunner and Performance Center, like many other testers. Unlike (alas!) so many other testers with those bullet points on the résumé, he’s not inclined merely to go through the motions and use tools for checking. He is an astute, passionate critical thinker, entirely focused on investigating and defending the value of the products for which he’s responsible by identifying problems that threaten that value. And yet’s he’s not of the Quality Assurance school; he entirely understands that assuring quality is the responsibility of those who produce and manage the work—programmers, writers, designers, artists, and managers. His job, as he sees it, is to make quality assurance possible. He collaborates with the project community, investigates the product ,and provides the most important, most timely information he can to the people who are producing and managing the work. With that, they can make the decisions they must make, informed by the very best technical information that he can provide.

He’s past President of the Association for Software Testing; he was Conference Chair for the 2009 Conference for the Association for Software Testing; he maintains one blog called Questioning Software and another called Is There A Problem Here?. He was recently given a three-part interview by UTest; you can read that here, and here, and here.

And, as of this writing, he’s available. For someone looking for a tester, he’s like the dream date that you spy across the dance floor whom a friend tells you is single, smart, modest, and well-off. The only issue is that maybe he’s a little too modest. He won’t be on the dance floor for long, because some organization will come along and sweep him off his feet. And that organization will be exceptionally lucky.

And that organization could be yours. His email address is ben at qualityfrog (period) com.

Postscript: Ben’s been hired by a company near Phoenix, AZ. You had your chance!