DevelopsenseLogo

Exegesis Saves! (Part 1)

This morning, I read a sentence that bugged me.

“In successful agile development teams, every team member takes responsibility for quality.”

I’ve seen sentences of that general form plenty of times before. Whether I’ve reacted or not, they’ve always bugged me, and today I decided to probe into why.

Rather than doing so on my own, I thought it would be more fun and more interesting to involve my community, so I posted a challenge on Twitter. If I want to get any other work done, I’m going to have to learn to stop doing that. I posted:

“Thinking Thursday. Test this sentence: ‘In successful agile development teams, every team member takes responsibility for quality.'”

Although I have great respect for my colleagues, I hadn’t anticipated so many interesting replies. So, today, my summary of the responses on Twitter. Soon, a brief conversation between James Bach and me, and shortly thereafter, my own assessment.

Questioning the Mission

Adam Goucher was the first to reply. He responded to my tweet by noting, “I could also check it by putting into a wordprocessor and seeing what is underlined.” I read this, laughed, and replied, “You could. But I ask for testing. :)” Adam promptly replied that he had thus clarified the mission. Yes; absolutely. One tactic for refining your mission is to make an offer. Acceptance, rejection, or other reactions to the offer may be revealing.

Before testing the sentence, Shrini Kulkarni immediately (and wisely) questioned the testing assignment. “First question: Why are you asking this question, and how will you use my answer? Question #2: What is your motivation for asking this question and how will you evaluate my response?”
Those were splendid questions. I emphasize, especially for all those new to testing: if you feel uncertain or confused, it’s vitally important to question the mission to reduce the risk that your assumptions don’t align with your client’s assumptions.

I answered. “I’m exploring what bugs me (and others) about the sentence. I’m not sure how (or even if) I’m going to use the information just yet; that’s part of my exploration.” That’s the why, my motivation. In answer to the evaluation question, I listed intellectual or emotional stimulation; insight, epiphanies, and reminders of old epiphanies as factors. I didn’t add then (but I will now) that I didn’t think of it as a competition. (To that end, I’ve presented as many of the replies as I’ve been able, and although I wasn’t intending to evaluate them in a competitive way, I can’t help but be impressed.)

Shrini had a couple more questions. “What is your interpretation of ‘testing’ of a sentence like this?” The goal, he said, was to know how my notion of testing the sentence might be different from his. At the time, I was away from my computer, so I didn’t respond to Shrini right away. So Shrini did something else that an expert tester does; if you can’t get an answer, state your assumptions. “One way I can think of ‘testing’ a sentence is to check if it is true. So my question would be “how do you [missing word] it is true?” (I believe the missing word was a typo.) “Another interpretation of testing a sentence—a linguistic construct—is to check if it is formed as per the rules of grammar.” So here Shrini, in the face of an ambiguous assignment, gave two possible interpretations and got to work, focusing on the one that he figured was the deeper problem.

Martin Jansson also asked for clarification: “When asking me to test this, are you the only stakeholder?” I was away when that message came in too. In reply, I’ll say now that I was the only client, but in a way, lots of people might be stakeholders. I leave it as an exercise to figure out who those stakeholders might be.

Questioning the Context of the Sentence

In addition to questioning the mission, questioning the context is crucial. Martin Jansson had a bunch of context questions. “Since Twitter limits only to 140 chars, can I see the full length of what was originally intended? Is this sentence localized and is originally from another language? Can I see the original one? What other sentences are connected to the one we are testing? What will the system and its environment look like?” Griffin Jones asked, “What is the history of this sentence in this context? Is this sentence the image we want to project as a company or group? Do our peers/competition make a similar claim? Do we publicly claim to follow that sentence?”

Identifying the Problems

The very first reply that I received was from Florin Duriou, who responded simply and directly, “too generic”. As Lynn McKee said, “Many words in that sentence are subjective.” I agree with that, so let’s get specific. How?

Shrini offered two approaches. “I do little of Jerry Weinberg’s “Mary had a little lamb” exercise to see possible meanings of every word and their combinations.” That exercise, from Exploring Requirements, involves going through the simple sentence “Mary had a little lamb,” and emphasizing each word in turn to see what other interpretations might be lurking.

  • “MARY had a little lamb” (but Joe didn’t, so he was jealous)
  • “Mary HAD a little lamb” (but she doesn’t any more)
  • “Mary had A little lamb” (then she had two—and now she has a flock)
  • “Mary had a LITTLE lamb” (but too much lamb would have been bad for her diet)
  • “Mary had a little LAMB” (why did you think I said “ham”?)

Shrini’s second approach: “Testing a sentence, I look for ‘words’ in it—like successful, agile, development, teams, team, member, quality, responsibility.” By putting “words” in quotes, I think Shrini was emphasizing that every one of these ideas are concepts, constructs, thought-stuff. So let’s look at some of those words and constructs in more detail.

“Successful”

Success is context-dependent. Griffin, Lynn, Peter Walen, and Stephen Hill all questioned the meaning of “success”. Simon Morley wanted to know whether we might deem a team successful without knowing what the team was doing or producing-and whether its efforts were desired. Both Pete Houghton and Martin asked whether criteria for success included shipping on time. Martin also asked if quality was the only factor that makes an Agile team “successful”. “What if they hate each other and learn nothing from what they do?”

“Responsibility”, “Agile”, “Team”

What does “responsibility” mean? Áine McGovern held that programmers should take responsibility for testing at low levels before testers get their hands on the product; that everyone is are responsible for a product that works well. Yet responsibility might mean “blame”, as both Peter Houghton and Peter Walen observed. What does “agile” mean? As Martin joked, “Did we mean Agile or perhaps AGILE? ‘Cause being AGILE is so much better than just lowercase agile.” Does the Agile part even matter? Anand or Komal Ramdeo ( I don’t know which; they have a joint Twitter account) noted that “agile” could be removed from the sentence without loss of meaning, if we believe that teams are successful when every team member takes responsibility. Pete Houghton thought to question what we mean by “team”. Griffin even questioned the role of “in” in the sentence.

Words in Combination

Joining all these words into a sentence leads to a kind of combinatorial explosion of possible interpretations. Lynn noted that success might not equal quality, depending on the project or organizational goals. I agree—and the project and organizational goals might come into conflict from time to time too, as the organization might have many projects on the go, and they projects might compete for resources.

Simon asked what “taking responsibility for quality” might involve or exclude. Peter Walen asked if we could infer that in non-agile teams, no team members are responsible for quality—or if successful waterfall teams didn’t need to worry about quality. Tim Western and had another variation: unsuccessful agile development teams don’t take responsibility for quality? Michel Kraaij asked pointedly, “So if the quality sucks, no one takes responsibility?” Adam Brown had yet another variation: wouldn’t every team member take responsibility for quality even if the project was unsuccessful?

Our sentence might be subject to the graveyard fallacy, a central theme of The Black Swan. The successful often attribute their success to some factor or practice or approach; the unsuccessful who used the same factor, practice, or approach but who were less lucky don’t survive to tell of their experience with the factor—and people seem disinclined to listen to the unsuccessful.  What can you learn from losers, after all? Nassim Nicholas Taleb, in The Black Swan, retells a story from Marcus Tullis Cicero, a tale of skeptical inquiry. “One Diagoras, a nonbeliever in the gods, was shown painted tablets bearing the portraits of some worshipers who prayed to survive the subsequent shipwreck. The implication was that praying protects you from drowning. Diagoras asked, ‘Where were the pictures of those who prayed, then drowned?’ The drowned worshippers, being dead, would have a lot of trouble advertising their experiences from the bottom of the sea. This can fool the casual observer into believing in miracles.” (You can keep reading here.) That’s what I thought of when I read Peter Houghton’s reply: “What about the unsuccessful teams where everyone also takes responsibility?”

“Quality”

Griffin asked for the definition of quality in this context. Markus Deibel asked if the notion of quality was based on each team member’s definition of quality, or on a common set of quality rules. Good questions; as Anand (or Komal) Ramdeo put it, “‘quality’ might have different meaning to different people—so everyone is responsible for quality but the product can still suck.” Quite right, I would argue—with the additional observation that the product might suck to some people and not to others.

Adam Goucher suggested replacing “quality” in the troublesome sentence with “the product”, as customers likely care more about that then the “quality”. I think Adam’s suggestion is to focus on something more concrete (the product), and less abstract. A good idea, but I think it shifts the problem rather than confronting it. Whatever people have to say about the product, their evaluation is an expression of a quality judgement. Martin suggested that in a successful agile development team, everyone would know that quality is abstract—and that its meaning is therefore not to be taken for granted. I agree (yet everyone? would?), and I’d be specific about this: quality isn’t an intrinsic and objective attribute of something. Instead, it’s a relationship between something and some person.

More Probing Questions

Griffin asked “compared to what?” A little while ago I jokingly suggested to Markus Gärtner that he write a macro that would at a keystroke issue the question “compared to what?” Zeger Van Hese reported that I had triggered his macro successfully. “Compared to what? Successful to whom? Quality to whom? Every time? Really?”

Zeger and Martin both raised interesting questions about equality and responsibility. Does every member of the agile team define quality equally? Can all team members ever be equally responsible for quality? Is it even possible to take responsibility for quality in the team? (For my part, I’ll speak to this issue later.) Zeger even raised what I will now call The Animal Farm Take on Responsibility: “I can’t help but thinking ‘…but some take more responsibility than others,'” he wrote.

The explicit talk of equality and implicit reference to power reminded me of Jerry Weinberg’s observation that decisions about quality are always political, made by people who have the authority—the political power—to make them.

Deciding What and How to Observe

Griffin emphasized a question that we must ask in addition to “compared to what?” He wanted to know “according to whom”. That’s important because observations, comparisons, and measurements are never value-neutral; they always depend upon at least one person, and what and how each person observes. “What would I see/hear/feel when a ‘team member’ ‘takes responsibility’?” Simon adds, “What would a team member /not/ ‘taking responsibility for quality’ look like?” Anand (or Komal) asks, “Is there any scale to measure responsibility? How do you compare more or less responsible?”

No Problem

Some people found no problem at all with the sentence. Alan Cooper retweeted it, adding “True”. Bill Clark responded to that, saying, “If the team looks good, everyone on it looks good. A bunch of lone coyotes running here and there not so much.” Ron Jeffries said, “The sentence is perfect. By the way, what did you want it to do?” An important question, but I might have placed it before my evaluation of the sentence’s perfection.

All of these responses were interesting and valuable. The most gratifying one, though, came from Zeger Van Hese, who raced to his blog before I could get to mine. You can read his response here.

More to come!

8 replies to “Exegesis Saves! (Part 1)”

  1. […] This post was mentioned on Twitter by Michael Bolton, Anand and Komal, Testing News, Luke Schubert, Ram (srm) and others. Ram (srm) said: RT @michaelbolton: Published: Exegesis Saves! A summary of yesterday's testing challenge, part 1. http://bit.ly/fLJusf #testing #softwa … […]

    Reply
  2. I’m not bugged about the sentence.
    I generally ignore AGILE stuff, and have made a habit of exchanging “Agile” with “good”.

    “In successful good development teams, every team member takes responsibility for quality.” is a bit clumsy and stating too much.
    I had preferred something like a Team Ownership Heuristic:
    It is good if team members care about their product, and take the responsibility they can.

    Reply
  3. “it’s vitally important to question the mission to reduce the risk that your assumptions don’t align with your client’s assumptions.”

    Yes!

    Just yesterday, I was in a team meeting along with several developers, QAers, Project Managers, etc. This was a weekly meeting for a time-critical project due in mid-March. Tensions are running very high, since virtually everyone is skeptical that we can actually meet that extremely-tight date.

    One of the developers talked about his design for one of the features. He just happened to mention that he decided to refactor a particular module, and that of course this means that other modules which interface with this one must be modified.

    Everyone else in the meeting looked stunned. It was clear that this developer had failed to realize the central, overriding factor of this project – a fixed end date. What he was proposing would require unplanned development and regression testing, and almost certainly blow the schedule by a significant amount.

    I had repeatedly been challenging the schedule assumptions in this same weekly meeting for months. And it was always asserted that mid-March was the date. He was at each of these meetings.

    Yet somehow this one developer hadn’t understood that message. In his mind, he was free to refactor any code he chose – whether it was needed or not. His assumptions were clearly not in alignment with anyone else’s.

    Reply
  4. Hello Michael — interesting question.

    I’m pleased by the questions in two ways – first that our test community has to skill to pin things down so well, even in a 140-character world – and second that, *I hope*, *I believe*, I knew you well enough to answer the question without having to clarify context.

    That said, here goes …

    I too, have struggled with this phrase. Reflecting on it, I can find three main reasons.

    First of all, you have the classic problem with responsibility: If we make quality one person’s responsibility, then there is an implicit excuse by everyone else to do shoddy work. (If memory serves I have read at least one major article about a company that created a test/QA team and defects increased, for just this reason.)

    On the other hand, making quality everyone’s responsibility means there is no “single wringable neck. (Compare that with the Scrum methodology providing a ‘single wringable neck’ for features in the form of the product owner.)

    So instead of everyone being responsible for quality, it’s a slippery slope to no one.

    Now, of course, the common response to that is the “No True Agilist” argument: After all, no /true/ Agile team would allow that kind of slogan to slip into barbarism. … right?

    Yet I have another, more subtle issue with the “every team member is responsible for quality” statement: It doesn’t tell you want to you

    Consider, for example, the team where you are a member of the tech staff and Bob is, as you see it, underperforming. What do you do? Do you talk him? Do you bring it up in front of the group?

    Does “responsible for quality” mean you test your own stuff more (as a developer?) how can an analyst be “responsible for quality?” What does that /mean/?

    If the statement is true, how then should we live?

    Of course, it’s possible the article explained that with examples, but, given your discomfort, somehow, I suspect that is not the case.

    Here’s the deal: “Everyone is responsible for quality” isn’t a guiding principle, it’s a /slogan/. Slogans, like “Together Each Achieves More”, or “Quality is Job #1” don’t actually help people do things in the moment.

    I’m not the first guy to observe this; Dr. W. Edwards Deming, the guru of manufacturing, lists “Eliminate Slogans” as point #10 of his fourteen points to save manufacturing:

    http://curiouscat.com/management/demings14points.cfm

    (Not everything Deming said applies to software, but I’m with him on this one.)

    Third, (and I think we’ve discussed this enough that we don’t need to keep harping on it), I’m a little uncomfortable with the declaration about ‘agile’ teams. Reading the sentence, it means that if you have specific people who focus on quality, you /aren’t/ agile, right?

    I don’t see that anywhere in the Agile Manifesto, so the statement seems to be using a very specific definition of ‘agile’ without ever clarifying it. Like ‘Quality’, ‘Architecture’, ‘Leader’, and ‘Maturity’, I get uncomfortable when these terms which we do not all agree on are used as if we /*do*/ have consensus.

    For that matter, the sentence uses Quality in the same way without defining it. Fancy that.

    Conclusion: The ideas behind the slogan might be concrete and helpful, but we don’t know — the slogan itself is lossy. Some people, especially those with a similar background and mindset, might get it intuitively, and, for them, as Ron said, it might be “perfect.”

    For the rest of us (or those who are simply curious), a statement like that begs for explanation, examples, and context.

    But that’s just me talking.

    Reply
  5. “In successful agile development teams, every team member takes responsibility for quality.”

    When I apply my various parsing macros to this sentence, I find it equivalent to the shorter sentence:

    “For an agile team to be good, every team member has to be good.”

    Duh!

    Reply
  6. In the responsibility part and from Pete Houghton’s reply, I see a problem:
    “the unsuccessful who used the same factor, practice, or approach but who were less lucky don’t survive to tell of their experience with the factor”

    As Jerry Weinberg points out, unlucky in this context means that management build up on luck rather than managing the project. There the word “unlucky” might mean that the project was managed too bad or not managed at all.

    There is another meaning in the word “lucky” or not so “lucky”, which might reveal its real intention. If by “lucky” you mean that there is some kind of magic ingredient in the mix, then you won’t know what that magic is, and whether or not it might disappear tomorrow. This is an instance of having the wrong model or explanation for an effect that you observed.

    The observation is “we got a successful project” (and I don’t divide between Agile and traditional here), but the cause may be too diverse for a human to oversee. Maybe we attribute on particular practice to this successful outcome, but we might have used the wrong model in doing so.

    You pretty well know that this is something that might happen for science as well as for systems thinking. Still I see more teams working successfully when everyone feels responsible for the “quality” – this ambiguous word which should be abandoned from management speak – than when not. This is a gut feeling, but I trust my trenches in this regard.

    Reply
  7. Hello and thank you for this interesting discussion.
    My experience in this is that you can exchange the word “quality” with “speak up”. “In successful agile development teams, every team member have the responsibility to speak up” and then make it a little longer “Everyone in the team is responsible to speak up when they see a potential problem or when there is a chance to improve something”.

    Michael replies: I don’t know about your context, but I speak up when I see a potential problem, irrespective of whether it’s an Agile team or not. My clients (programmers, managers, other stakeholders) are welcome to note the suggestion or the risk, and deal with it, or ignore it. If I’m ignored too often, I’ll consider whether the information that I’m supplying is informantion that my stakeholders want. If it isn’t, I adjust that information, or I leave. But Agile-ness has nothing to do with that, really.

    That gives everyone a hands-on instruction what to do when you find something that needs attention and if you encourage ALL your team member to speak up when they see something and show them that their opinions count, then you can find problems early in the project and also get good suggestions for solutions on how to solve the problems, in other words make a better product.

    The agile principal says that in a team, all members are equal and the same (more or less). But in reality people are different and are good at different things and therefore have different opinions in what matters. It does not mean that action needs to be taken to every issue raised, just that if all in the team including the tester raise issues you are more certain to reach a higher “quality” of the project.

    I don’t believe that all team members are equal, and I don’t need to believe that to be effective. I don’t have equal authority to change the code. No problem; that’s not my role, and that’s okay with me.

    To use an ambiguous word like “quality” in an instruction or statement could result in a discussion like “what is quality?” or make people uncertain if the issue they found is related to the term quality.

    I’m not afraid of discussions like “what is quality?” To me, quality is value to some person. What makes people afraid, perhaps, is that discussions and decisions about quality—as Jerry Weinberg points out—are always political and emotional. That is, the decisions about quality are made by people with the power to make them, with the goal of providing or maintaining value to people who are perceived to be important. And those discussions and decisions aren’t entirely rational, either; they’re not based on what’s so, but on how people feel about what’s so. What’s to fear there? Some people seem uncomfortable with the idea that we might have to sort out whose perception of quality matters, and whose perception doesn’t matter so much. We might have to deal with the idea that some people are more important than others. We might have to understand that we’re here to serve organizations and businesses, and that those entities are making decisions all the time about whose values matter and whose don’t. That’s a constant in human relations, and I don’t see it as anything to worry about. We talk it over, and we sort it out.

    A bug is something that threatens the value of the product. An issue, to me, is something that threatens the value of the project, or the testing effort, or the business. So I don’t have a particular problem with that either: if I see something that I believe might theaten value, I report it, and explain why I perceive that it might threaten value.

    Reply

Leave a Comment