Blog: Testers: Get Out of the Quality Assurance Business

The other day, Cory Foy tweeted a challenge: “Having a QA department is a sign of incompetency in your Development department. Discuss.”

Here’s what I think: I’m a tester, and it’s time for our craft to grow up. Whatever the organizational structure of our development shops, it’s time for us testers to get out of the Quality Assurance business.

In the fall of 2008, I was at the Amplifying Your Effectiveness conference (AYE), assisting Fiona Charles and Jerry Weinberg with a session called “Testing Lies”. Jerry was sitting at the front of the room, and as people were milling in and getting settled, I heard him in the middle of a chat with a couple of people sitting close to him. “You’re in quality assurance?” he asked. Yes, came the answer. “So, are you allowed to change the source code for the programs you test?” No, definitely not. “That’s interesting. Then how can you assure quality?”

A good question, and one that immediately reminded me of a conversation more than ten years earlier. In the fall of 1996, Cem Kaner presented his Black Box Software Testing course at Quarterdeck. I was a program manager at the time, but the head of Quality Assurance (that is, testing) had invited me to attend the class. As part of it, Cem led a discussion as to whether the testing group should really be called “Quality Assurance”. His stance was that individuals—programmers and testers alike—could certainly assure the quality of their own work, but that testers could not assure the quality of the work of others, and shouldn’t try it. The quality assurance role in the company, Cem said, lay with the management and the CEO (the principal quality officer in the company), since it was they—and certainly not the testers—who had the authority to make decisions about quality. Over the years he has continued to develop this idea, principally in versions of his presentations and papers on The Ongoing Revolution in Software Testing, but the concept suffuses all of his work. The role for us is not quality assurance; we don’t have control over the schedule, the budget, programmer staffing, product scope, the development model, customer relationships, and so forth. But when we’re doing our best work, we’re providing valuable, timely information about the actual state of the product and the project. We don’t own quality; we’re helping the people who are responsible for quality and the things that influence it. “Quality assistance; that’s what we do.”

More recently, in an interview with Roy Osherove, James Bach also notes that we testers are not gatekeepers of quality. We don’t have responsibility for quality any more than anyone else; everyone on the development team has that responsibility. When he first became a test manager at Apple Computer way back when, James was energized by the idea that he was the quality gatekeeper. “I came to realize later that this was terribly insulting to everyone else on the team, because the subtle message to everyone else on the team is ‘You guys don’t really care, do you? You don’t care like I do. I’m a tester, that means I care.’ Except the developers are the people who create quality; they make the quality happen. Without the developers, nothing would be there; you’d have zero quality. So it’s quite insulting, and when you insult them like that, they don’t want to work with you.”

Last week, I attended the STAR East conference in Orlando. Many times, I was approached by testers and test managers who asked for my help. They wanted me to advise them on how they could get programmers to adopt “best practices”. They wanted to know how they could influence the managers to do a better job. They wanted to know how to impose one kind of process model or another on the development team. In one session, testers bemoaned the quality of the requirements that they were receiving from the business people (moreover, repeating a common mistake, the testers said “requirements” when they meant requirement documents). In response, one fellow declared that when he got back to work, the testers were going to take over the job of writing the requirements.

In answer to the request for advice, the most helpful reply I can give, by far, is this: these are not the businesses that skilled testers are in.

We are not the brains of the project. That is to say, we don’t control it. Our role is to provide ESP—not extra-sensory perception, but extra sensory perception. We’re extra eyes, ears, fingertips, noses, and taste buds for the programmers and the managers. We’re extensions of their senses. At our best, we’re like extremely sensitive and well-calibrated instruments—microscopes, telescopes, super-sensitive microphones, vernier calipers, mass spectrometers. Bomb-sniffing detectors. (The idea that we are the test instruments comes to me from Cem Kaner.) We help the programmers and the managers to see and hear and otherwise sense things that, in the limited time available to them, and in the mindset that they need to do their work, they might not be able to sense on their own.

Listen: if you really want to improve the quality of the code and think that you can, become a programmer. I’ve done that. I can assure you that if you do it too, you’ll quickly find out how challenging and humbling a job it is to be a truly excellent programmer—because like all tools, the computer extends your incompetence as quickly and powerfully as it extends your competence. If you want to manage the project, become a project manager. I’ve done that too, and I was pretty good at it. But try it, and you’ll soon discover that quality is value to some person or persons who matter, and that your own standards of quality are far less important than those who actually use the product and pay the bills. Become a project manager, and you’ll almost immediately realize that the decision to release a product is informed by technical issues but is ultimately a business decision, balancing the value of the product and bugs—threats to the value of the product—against the costs of not releasing it.

In neither case would I have found it helpful for someone who was neither a programmer nor a project manager to whine to me that I didn’t have respect for quality; worse still would have been instruction on how to do my job from people who had never done that kind of work. In both of those roles, what I wanted from testers was information. As a programmer, I wanted to know about problems the testers had found in my code, how they had found them, the ways in which the product might not work, and the steps and clues I needed to point me towards finding and fixing the problem myself. As a program manager, I wanted to know what information the testers had gathered about the product, and how they had configured, operated, observed, and evaluated the product to get that information. I wanted to know how our product worked differently from the ways in which it had always worked. I wanted to know about internal inconsistencies. I wanted to know how our product worked in comparison to other products on the market; how it was worse, how it was better. I wanted to know how the product stacked up against claims that we were making for it.

So:  you want to have an influence on quality, and on the team.  Want to know how to influence the programmers in a positive way?

  • Tell the programmers, as James suggests in the interview, that your principal goal is to help them look good—and then start believing it. Your job is not to shame, or to blame, or to be evil. I don’t think we should even joke about it, because it isn’t funny.
  • You’re often the bearer of bad news. Recognize that, and deliver it with compassion and humility.
  • You might be wrong too.  Be skeptical about your own conclusions.
  • Focus on exploring, discovering, investigating, and learning about the product, rather than on confirming things that we already know about it.
  • Report what you’ve learned about the product in a way that identifies its value and threats to that value.
  • Try to understand how the product works on all of the levels you can comprehend, from the highest to the lowest.  Appreciate that it’s complex, so that when you have some harebrained idea about how simple it is to fix a problem, or to find all the bugs in the code, you can take the opportunity to pause and reflect.
  • Express sincere interest in what programmers do, and learn to code if that suits you. At least, learn a little something about how code works and what it does.
  • Don’t ever tell programmers how they should be doing their work. If you actually believe that that’s your role, try a reality check: How do you like it when they do that to you?

Want to know how to influence the managers?

  • Provide them with the information they need to make informed decisions, and then let them make the decisions.
  • Remain fully aware that they’re making business decisions, not just technical ones.
  • Know that the product doesn’t necessarily have to hew to your standard of quality.
  • It’s not the development manager’s job, nor anyone else’s, to make you happy. It might be part of their job to help you to be more productive.  Help them understand how to do that.  Particularly draw attention to the fact that…
  • Issues that slow down testing are terribly important, because they allow bugs the opportunity to hide for longer and deeper. So report not only bugs in the product, but issues that slow down testing.
  • If you want to provide information to improve the development process, report on how you’re actually spending your time.
  • Note, as is so commonly the case, why testing is taking so long—how little time you’re spending on actually testing the product, and how much time you’re spending on bug investigation and reporting, setup, meetings, administrivia, and other interruptions to obtaining test coverage.
  • Focus on making these reports accurate (say to the nearest five or ten per cent) rather than precise, because most problems that we have in software development can be seen and solved with first-order measures rather than six-decimal analyses derived from models that are invalid anyway.
  • Show the managers that the majority of problems that you find aren’t exposed by mindless repetition of test cases, but by actions and observations that the test cases don’t cover—that is, by your sapient investigation of the product.
  • Help managers and programmers alike to recognize that test automation is more, far more, than programming a machine to pound keys.
  • Help everyone to understand that automation extends some kinds of testing and greatly limits others.
  • Help to keep people aware of the difference between testing and checking.  Help them to recognize the value of each, and that excellent checking requires a great deal of testing skill.
  • Work to demonstrate that your business is skilled exploration of the product, and help managers to realize that that’s how we really find problems.
  • Help the team to avoid the trap of thinking of software development as a linear process rather than an organic one.
  • Help managers and programmers to avoid confusing the “testing phase” with what it really is: the fixing phase.
  • When asked to “sign off” on the product, politely offer to report on the testing you’ve done, but leave approval to those whose approval really matters: the product owners.

Want to earn the respect of your team?

  • Be a service to the project, not an obstacle. You’re a provider of information, not a process enforcer.
  • Stop trying to “own” quality, and take as your default assumption that everyone else on the project is at least as concerned about quality as you are.
  • Recognize that your role is to think critically—to help to prevent programmers and managers from being fooled—and that that starts with not being fooled yourself.
  • Diversify your skills, your team, and your tactics. As Karl Weick says, “If you want to understand something complicated, you have to complicate yourself.”
  • As James Bach says, invent testing for yourself. That is, don’t stop at accepting what other people say (including me); field-test it against your own experience, knowledge, and skills. Check in with your community, and see what they’re doing.
  • If there’s something that you can learn that will help to sharpen your knowledge or your senses or your capacity to test, learn it.
  • Study the skills of testing, particularly your critical thinking skills, but also work on systems thinking, scientific thinking, the social life of information, human-computer interaction, data representation, programming, math, measurement
  • Practice the skills that you need and that you’ve read about. Sharpen your skills by joining the Weekend Testing movement.
  • Get a skilled mentor to help you. If you can’t find one locally, the Internet will provide. Ask for help!
  • Don’t bend to pressure to become a commodity. Certification means that you’re spending money to become indistinguishable from a hundred thousand other people. Make your own mark.
  • Be aware that process models are just that—models—and that all process models—particularly those involving human activities like software development—leave out volumes of detail on what’s really going on.
  • Focus on developing your individual skill set and mindset, so that you can be adaptable to any process model that comes along.
  • Share your copy of Lessons Learned in Software Testing and Perfect Software and Other Illusions About Testing.

Ultimately, if you’re a tester, get out of the quality assurance business. Be the best tester you can possibly be: a skilled investigator, rather than a programmer or project manager wannabe.

Note: every link above points to something that I’ve found to be valuable in developing the thinking and practice of testing. Some I wrote; most I didn’t.  Feast your mind!

Want to know more? Learn about Rapid Software Testing classes here.

93 responses to “Testers: Get Out of the Quality Assurance Business”

  1. […] for this team; they can sit together. Testers aren’t involved (yet) — I talked with the QA manager (who is trying to manage the day to day work of testers on two agile projects plus a few […]

  2. […] and a “Quality Assurance Manager.” We all know now, or should, that those roles have nothing to do with quality assurance. I didn’t write feature code as a QA analyst and I didn’t dictate to programmers how to […]

  3. Miguel says:

    I think Quality Assurance may be a good name to a team in some cases, at my company the testing team have different responsibilities besides testing software such as document analysis, questioning development processes, software-related legal issues delegated to software, software-related political issues, etc… What name would you think for a department like this one?

    I understand you could think that the main problem here is the responsibilities taken by the testers but let’s face it, if it is what it is, and you only can switch the name, which would it be?

    Michael replies: Let’s make it clear that I can’t—and I shouldn’t—enforce orthodoxy, so this is an opinion, not a statement of dogma or best practice.

    You could call the testing group “quality assistance”, or “quality awareness”, if you want to keep the QA abbreviation. Consider that programmers do all of the things that you mention above, at least to some degree. Programmers do more than programming, but we still call them “programmers”. So it seems to me that “testers” could do all that stuff, and you could still call them testers. Why not call them “testers”?

    The point of the post is not really about the label; it’s about the idea. Whatever else testers might do, they don’t assure quality. They do provide information to the people who DO assure quality.

  4. […] do you think? Is this topic important for testing? Should we treat is in the way Michael Bolton advises testers with respect to “Quality Assurance Business” (spoiler alert/hint: He advises to stay out of […]

  5. […] ? ?????????, ????????? ?????? ?????? ??????? – “Testers: Get Out of the Quality Assurance Business”. ??? ?????? ?? ???????. ??????? ????? – […]

  6. Kyo says:

    From my personal experience what developers tend to appreciate most of all it’s the fact that even when you’re a QA you can talk with them at the same level about the code, I use to be a former programmer a really good one, but because of a bad experience in a previous job (lack of creative ways or innovation paths to develop new features due to manager restrictions) I decided to switch to QA and what I have found it’s that most of the teams ussually start the project giving me the evil eye “grrrr that qa guy I don’t like him…” but when they start seeing me on the code reviews exposing my personal opinions on ***what are the best ways to implement something from a QA perspective***, they see the tools that I program to automate the several applications, or even libraries for commercial or open source automation/performance solutions and last but most importante they see a change on how I report the bugs that I found, I can’t stand out this enough but every QA should know where in the code is the defect that they just found, so you can report in an accurate way to the programmers and minimize the resolving time a great deal, it’s not the same when you say “login is failing when I type blah blah blah” instead of saying “the XXXX field on this function/method/class is throwing THIS exception when I give this input/s”.
    Then you have their respect and they’ll treat you as an equal that is helping them to develop a better software, and ofcourse you’ll have a direct impact in the project schedule and quality being a member that any team will want on their line up.
    Ofcourse is a must to be polite and always treat the code that those developers have spent hours of creative thinking to create with the respect that deserves as stated on this article.
    I disagree with the idea that if you know how to code and have opinions about the source of your current project you should move to the dev team, I’m more of the opinion that the days of the black box testing should be done and buried QAs need to know the code that they’re testing and the techonolgies that are being used to create it in order to provide even more information to the managers that will make the call to deliver the product or not.

    Michael replies: I agree that it’s a good idea for some testers, at least, to understand technology at the code level. And it’s a good idea for testers to understand the technologies that they’re testing. I’ll go further: often testers develop knowledge about the technologies that are being developed. But I don’t agree that “the days of black box testing should be dead and buried”. I fine with not seeing the code of things that I’m testing. It affords a different perspective that can be especially valuable when everyone else is focused on the code. If you believe that every tester must know software on a coding level, you’d have to agree that every tester should also understand design, usability, security, performance, diagramming, systems thinking, critical thinking, all domain knowledge, and should have a complete grasp of how to write grammatically correct English sentences. Otherwise you’d you’d have to explain why those things aren’t important. As alternative, perhaps you’d acknowledge that a really good development team consists of a number of people with diverse talents and skills working together, and that it’s okay for a tester not be a coder when the tester brings other skills and perspectives to the table.

  7. Baby Bunny says:

    Wow and thank you for your time to create this thread I am finding it invalueable. At present I’m seeking a career change and am a zygote within the IT arena, if its ok to label it that. What you have stated applies to all industries in a transferable skills/idea’s set way. That little box might be green this side however the opposite side might be red, oh wait… your colour blind and that box is actually big, now its not even a box… wow how can I see what in it if its not a box? hmm
    I guess sometimes we get so focused on the problems we forget to focus on their solutions. I’m going to look at what developers do within this industry to see if my skills can help there, thanks again for the reading tools.

  8. […] that I’m advocating to getting into Quality Assurance business as Micheal Bolton has very strong arguments against it. But that the ownership of Quality is shared amongst the team rather than just the […]

  9. […] online forum of some sort. Also, I would like to add that Michael Bolton has (as in most topics), covered this specific topic much more extensively than I, and is a great source of information in […]

  10. Pat Barnes says:

    I like a lot of what you said. Especially your recommendations to testers. But I think you have your “assure” and “ensure” mixed up. Quality Assurance doesn’t own, let alone “create” quality, but it provides confidence to the consumer of the product or results of a process that it is of a particular measure of quality (e.g., no sev 1 defects, or no salmonela poisoning).

    Michael replies: One of these days I’ll have to do a long post on the confidence issue. For now, here’s a short one (along with some longer comments).

    As testers, we cannot report that there are no Severity 1 defects in the product. At best, we can report that we have found no Severity 1 defects in the product, but we cannot achieve complete coverage, we are not omniscient, and we are not perfect. As with salmonella, you can’t observe the product (patient) and find evidence of no salmonella; you can only fail to find evidence of salmonella.

    Bugs don’t necessarily present immediately. Some do, but others remain latent for some time. Salmonella doesn’t present symptoms immediately, but over several hours, and the incubation period may be as long as a day. Observing your lunchtime customers doesn’t tell you in time for dinner that you’ve got clean prep surfaces. Even by lunchtime today, you don’t know that everything was perfectly clean yesterday; only that your customers have not developed an infection—but perhaps that’s because their immune response is very robust, and they were not exposed to enough contamination for illness to develop.

    The incubation period for discovering a bug may be even longer—years. Software is a domain that lives in the world of black and grey swans. We cannot present reasons why our clients can be confident, but we can show (or fail to show) reasons why they could doubt their confidence. Excellent testing and test reporting depends on understanding the extent and limits of what we know and what we can’t know.

  11. Don O'Neill says:

    Title:
    Software Assurance Trustworthiness and Rigor
    http://youtu.be/N2lT84hEhkU

    Description:

    Trustworthiness requires a commitment to rigor in both software production and its verification. No soft skill, rigor has a hard edge. In an environment of neglect and unmet need, the introduction of rigor would be a game changer, one that requires both cultural change and engineering know how with pass/fail training certification.

    Software Assurance only has meaning in the context of trustworthiness, that is, worthy of being trusted to fulfill the critical requirements needed for a particular software component, system, or system of systems. Software Assurance demands two capabilities associated with trustworthiness, the capability to produce trustworthy software products and the capability to verify that software products are trustworthy. Each depends on engineering and technology rigorously applied.

    The kernel of the layered defense approach to Software Assurance is Build Security In and Structured Programming with its rigorous and provably correct use of zero and one predicate prime programs along with proper programs composed of multiple prime programs limited to single entry and single exit.

    The layered defense approach described here includes Defense in Depth, Build Security In, Risk Management Calculation, and Resilience processes, engineering, and tool automation dependent on an elevated state of software engineering rigor and precision. There is much room for improvement in the Software Assurance methods and practices that assure such rigor.

    Michael replies: Just so I’m clear on this… you’re saying something about rigor, right?

    If so, you might be interested in this: http://www.satisfice.com/presentations/rigor.pdf

  12. Pat Barnes says:

    I hear what you are saying, but let me try another approach. I buy stuff. Mostly electronic stuff and cars, and my wife buys everything else. I rely heavily on Consumer Reports. I like the testing that they do and the fact that they are independent of the folks who are trying to sell me stuff. They provide me with “quality assurance.” They do nothing to actually “ensure” the product has quaility, but the fact that they are looking at it in some detail, and measuring it against some benchmarks, e.g., reported defects by readers in a survey, provides me with more assurance that a product will be reliable–or more reliable than some competing product–than if I just pick the pretty one. What they do is testing (and data gathering). What they provide to me the consumer or stakeholder is QA. It is not a lot of other things. It is not a garnatee that there are no defects. That’s not what I said. That’s not QA anyway, and your thesis isn’t no defects, it’s that testing is not QA. Consumer Reports can’t tell me that my new Honda has no defects. But they can provide me with some assurance that If I buy the Honda, I’m likely to get less defects than if I buy the Yugo. This is what Testers do for the customer. Confidence is also not binary. So in fact when CR tells me about their extensive testing process, and why the recommend the Honda, they do in deed provide reasons why I can be confident. Which is why I just bought my wife a new Honda.

  13. Angelos Mavrogiannakis says:

    Mike is very interest article. However in my view you cover the article from the “Role” point of view. What about the QA as a skill? A tester must have QA skills that will hep him to build better Test plans , Test cases etc,, So I find it a little dangerous as novice users may misread that they do not have to worry about QA as a skill.

    Michael replies: To me, quality assurance is not a skill. Skills do enable you to assure the quality of your own work to some degree, though.

  14. […] as a service to project stakeholders, which is echoed by Michael Bolton in his blog article titled Testers: Get Out of the Quality Assurance Business. In the article Bolton implores testers to recognise that they don’t solely “own” […]

  15. shrirang says:

    Very interesting blog. Please add me to your mailing list. Thanks a lot for sharing.

    Regards,
    Shrirang

  16. Mr. Test says:

    I’ve been doing “QA” for well over 20 years. If I did not test the software, there would be little quality in many cases. I have to urge programmers to make changes to the software, not only because of bugs, but because of usability issues. If I did not URGE programmers who often have massive egos about the quality of their work, to fix or change the way that something does or does not work, most of it would not get done. So, I am assuring quality even though I am not the initial creator of the software. I’ve found that most programmers have little idea about what end users experience and are often detached from that reality. Without quality assurance testing and analysis, there is little quality. Quality Assistance? …maybe, but not just a tester.

    Michael replies: I notice that you call yourself “Mr. Test”. I wonder what “Mr. Develop” and “Mr. Manage” might say about the idea that they are detached from reality, or about the idea that you and you alone are “assuring” quality. I’m also curious about what you think being “just” a tester means.

  17. Jackie says:

    I moved from dev to qa about 4 years ago.
    My experience:
    QA people substitute the knowledge, the competence with the best practices etc.
    QA people think, they are working independently.
    They are not and they should not.
    QA people hate anyone who can actually do the job. I can read and write software. They sense it on the day one.
    I am well hated from the day one. Have been in this situation, at least 3 times.

    Michael replies: I’d like to offer this question, gently: Are you sure that differences in technical skill represent the only explanation for what sounds like a more complex social problem? The one and only explanation? “Hate” is a pretty strong word.

  18. Kevin says:

    While I agree with a great many of your assertions I think the point is being missed. There is a role for Quality Assurance vs. Quality Control. Testing is part of Quality control and is the assessment of the quality of the software relative to the requirements. Quality Assurance is the monitoring of the processes that assure the likelihood that the software is produced correctly and may be maintained in the future via production of deliverables (business case, requirements docs, design, test plans/cases etc.). They are different disciplines and responsibilities.

    Michael replies: I think you meant to say, “To me, testing is part of quality control…” To me, that (and the sentences that follow) require us to think of testing in terms of confirmation of what we hope or believe to be true. To me, that’s a highly oversimplified view of testing. The whole quality-control/quality assurance model comes from relatively ancient and not-very-helpful manufacturing models that don’t fit very well with software development. How would you distinguish between “quality assurance” and “management” in your description?

    Too many organizations do not recognize the difference and lump the two roles together. Most managers couldn’t describe how the two areas differ either. I’ve yet to meet a CTO/CIO/CEO that had a clue.

    Maybe the inability of people to distinguish or describe the distinction between the two roles suggests that the distinction isn’t very helpful.

    Just my 2 cents.

  19. Actually, in general, I find myself agreeing with most of what is presented here. With one exception:

    From my experience working in small teams, very small businesses, and startups, you’re not going to get very far or be able to offer much value, with the mindset that someone in authority needs to instruct you about what’s important, and inform you as to your specific responsibilities, in terms of product quality.

    Michael replies: I agree. But did I say anything like that, or is that something that you read into the piece?

    In my world, we work in service of a business. I don’t need instruction in what’s important, but I also don’t presume that I know everything that’s important to the business; nor do I presume that the business knows about important things I might be able to reveal. We all (including the business people) start with ideas about what’s important, and we learn as we go.

    When it comes to building a quality product, hierarchical relationship structures are regressive, unresponsive to change, vulnerable to more flexible shops, and mostly unwanted in a small shop, or startup environment. Nobody wants a mere “worker”, and the old model of pyramidal authority is going the way of the buckle-shoe.

    Perhaps. But I think you’re changing the subject here; I didn’t say anything about a “mere worker” or “pyramidal authority”. What I am saying is that testers should not usurp the roles of the programmer or the product owner, but support them.

    What IS wanted, is a colleague. Someone capable of negotiating, debating, discussing, and yes, personally investing in not just the “testing” or “quality”, but the user experience, the marketability, the engineering and design, and the financial success of the product.

    They want someone who *thinks* about, and *cares* about, the product. Because they do too. And they don’t want to spend their day working along side someone who only cares about “providing a service”.

    It’s surprising to me that you would consider providing a service and caring about the product as being inconsistent with each other. Thinking about and caring about the product is an important part of informing the service that I provide.

    This means, I spend my day doing more than just ferreting out and reporting on functional bugs in professionally written reports (though it is the majority of the work). I also talk to product and design about useability issues I encounter. I discuss with the CTO and sales folks ideas I have for encouraging up-sales in various interactions within the product. I work with developers to actually *fix* some of the bugs I find, but also to learn about their design and implementation decisions, and to discuss alternatives.

    Yes. Can you point out where I’m suggesting otherwise?

    Also, if I did any of these in a manner you describe above in your bullet-points (making developers cry, telling them I know their job better than they do, making unreasonable demands of management, treating people smugly), I would be fired in very short order. Nobody wants to work with an a$$hat.

    Yes. That was exactly the point I was making,

    But my experience has also been, that if I *don’t* engage beyond the level of a mere “service provider”, who is simply following the marching orders given to him by the superior who “actually owns quality”, then I’m also basically doing nothing more than the minimum required to fill the role, and that’s not at all what is expected. Eventually, that will get you fired, too.

    Nobody said anything about not engaging beyond the level of a mere service provider, or about “simply following the marching orders given to him by the superior”. That’s stuff you made up.

  20. An amazing read, I’ve moved from development to QA and needed to know what really QA is, well it’s just testing 😛

  21. Rob Huber says:

    QA IS different from testing. Testing is more common in Waterfall organizations. It happens after-the-fact, after design has been made and is a separate stage in which core functionality is allegedly set to work/scoped.

    Michael replies: Interesting; that’s a view of testing that, in our community, is as dated as Waterfall itself. For instance, in Rapid Software Testing, we maintain that testing is “evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.” Does this happen “after-the-fact, after design has been made, in a separate stage”? Not by our reckoning, since “‘experimentation’ implies interaction with a subject and observation of it as it is operating, but we are also referring to ‘thought experiments’ that involve purely hypothetical interaction. By referring to experimentation, we are not denying or rejecting other kinds of learning; we are merely trying to express that experimentation is a practice that characterizes testing. It also implies that testing is congruent with science.” (You can read about this at http://www.satisfice.com/blog/archives/856)

    Now, you may say, “But that’s redefining testing.” We would disagree with that, too.

    QA is more involved. The QA is more of an guide to the team – the subject matter expert who encourages the team to move towards Best Practices for Quality. What is different and separates a QA from a Tester will be involvement in the Team. Typically Testers are less involved – do not get involved in decisions early on and accept products as they come. QA is involved early in inception of these products and can guide the team toward goals of better Quality.

    Here’s at least one problem with that: what gives “the QA” his or her stature? What gives “the QA” his or her authority? Quality is multi-dimensional; “value to some person(s) who matter”. Without stature (presumably derived from expertise and experience in several domains), there’s no good reason to listen to someone who protests about some dimension of quality or another. And without authority, one could view “the QA” as a wise coach (at best, assuming stature) or as an inexperienced, irritating pest. With authority, “the QA”‘s role would be indistinguishable from management, as I suggest in the post.

    I agree with all comments on this post except that QA and Testing are the same thing. Great post, by the way.

    Thank you for the compliment.

  22. […] There is one post you need to read on the topic by Michael Bolton, Testers get out of the QA business. […]

  23. Connor Roberts says:

    TL;DR = 1) There’s no reason testers can’t make code commits, if it helps them become better testers. 2) The people with the most product knowledge/business risks should make the release decisions, regardless of job titles, so perhaps we have different experiences with what “management” is or is not.

    Michael replies: TL;DR 1) There’s no reason a goalie can’t try to score goals either, if it helps them becomes better goalies AND if they and their team are willing to leave the net unattended. 2) The people with business responsibility and authority should make business decisions, and releasing a product is a business decision, one informed only partly by technical considerations.

    1) On the quote of “quality assurance”:
    I take exception to the initial question asked in that post though (where the QA is asked if “they are in quality assurance” and are “allowed to change the source code”. I think the logic of both parties in the conversation are flawed. The asker is assuming that you can only “assure” quality if you have direct code/change access, which is not true, while the tester is saying “of course not” to being able to edit the code, which also does not make sense to me. There no reason a tester cannot be technical enough to help the actually team make code commits. That might be part of their process to become a better tester, and saying “of course not” comes across as avery close-minded on the tester’s part. That is a mind, content to live in a box.

    You can only assure quality if you have authority or control over the work. An investigative reporter can assure the quality of her own work, but she cannot assure the quality of the political process or the politicians she’s investigating. She can run for office, but at that point she is no longer in an investigative reporter’s role. When a tester is helping the team with code commits, the tester is not in the testing role at that moment. I’m not quite sure why this is so hard for people to understand (but I can’t assure the quality of their understanding). Perhaps the problem is that people confuse “task” and “role”, or that they misunderstand the notion of roles altogether. Here is an attempt to explain that.

    In some contexts, a tester who helps do code commits every now and then might be a integral part of their development process and of that tester’s own workflow to become more masterful at their skill-craft of testing. I do not believe in a hard division/lines drawn in the sand of ‘this is what a tester does’ and ‘this is what a dev does’. Do I think there are guard-rails that govern those roles? Sure, but there’s no way we can claim that context is key, and then say in all cases ‘a tester should never focus on this or that’.

    It is of the essence of being a tester to focus on testing, just as it is the essence of being a flight attendant not to focus on attending passengers on piloting, and just as it is of the essence of being a pilot to focus on piloting rather than attending passengers.

    2) On the subject of release ownership (this section is based on working in environments where product owners are part of the team, embedded, and have all the business knowledge/risk information to make release decisions):

    I also used to be empowered by believing QA was a ‘gatekeeper’ but they are not.

    Meaning that your “empowerment” was fake—right?

    The whole team owns the quality of the product, as the team is made up of other smart professionals too. This is why I believe the tester works in conjunction with the PO/team to determine if something is a show stopper or not. I think we’re in agreement on that based on your article, but the implementation of it perhaps not.

    I don’t understand that last clause, there.

    You quote about project management saying that a release is a business decision is true. Sometimes we intentionally release with know bugs because we have calculated the risks and deemed it acceptable. We might do a patch later or work those bugs into upcoming sprints, but we’re ‘OK’ to release now in the current state. I get that, that must be done.

    There’s a fine line there too though. If we agree that the whole team (scrum team: Scrum Master, Product Owner, Devs, Testers) are in charge of the quality, then to me it’s a logical extension of that thought to say the team gets to decide what is/isn’t ready for production. Remember, the team includes the product owner, who understands the business risks and justifications for or against releasing something with known defects. We keep saying that testers cannot assume they know about all the risks, which they don’t, but part of being a good tester is also becoming a subject matter expert in the product, so you can make much more informed testing decisions. I am thrilled when the tester, in a backlog grooming session, is able to educate the product owner on some business rules or development risks that they did not know, thus changing the scope of the release. This also means the product owner has some learning to do, but that collaboration between the two can foster that. I think it is key that testers be product experts as well as testing experts, as one fuels the other (again, this is within the context of website software development). This again ties into my lack of interest in drawing hard lines in the sand on what a Dev is, what a tester is, what a PO is. If we share the responsibility of quality on the team, then we also share the other things as well.

    Curious to hear your thoughts on this, and it is fine if we disagree, but if you look at forward-thinking Agile shops like Spotify, they foster team autonomy by giving them control over the release approval process. Management helps remove impediments, mentor, guide, etc, but I would disagree that a good management structure involves someone outside of the team saying yes/no on a release as a practice. That should be an outlier in my opinion and the final say should lie with a discussion between tester, dev and product owner who understands the business ramifications. The most successful models I have seen are when the product owner (who is embedded within the team) ultimately decides the go/no go.

    You do understand that “product owner” is a management position in the sense that I’m talking about, right? That the product owner is responsible for management decisions on the team?

    Note: I am cognizant that my understanding of these things is in constant evolution. As a learner, I am open to being convinced otherwise.

  24. Connor Roberts says:

    Thank you for the reply. Clarifications below.

    On gatekeeping: Yes, you could say my empowerment was “fake”. I think a more appropriate description of it would be ‘subconscious wishful thinking’ at the time, which I felt into because I wanted to combat the stigma of testers traditionally not being as valued as Devs.

    On Product Owners: I am speaking to my experience when I talk about Product Owners, and perhaps we misuse that title here, but they are embedded within the team and do have the final call on what is/isn’t going to production, but they themselves are not considered “management”. We are using different vernacular there, so this may just be a hangup on the semantic of the “management” title.

  25. […] quality. You just can’t. This is not a new topic, Michael Bolton wrote his popular blog post Testers Get Out of the Quality Assurance Business* over 5 years ago, and it was not a new topic […]

  26. […] of the product is archaic. Michael Bolton better frames this point in his blog post here Testers: Get Out of the Quality Assurance Business. Understanding why a given product is or is not getting attention can greatly help you do your job […]

  27. […] product risks. Here’s a good blog post by Michael Bolton better exploring that tangent Testers:  Get Out of the Quality Assurance Business «  Developsense Blog. Again, the purpose of testing is “to cast light on the status of the product and it’s […]

  28. […] So I want to close with statement which is the title to a blog that just came to my mind, while I finished my article. Testers get out of the Quality Assurance business! […]

  29. […] Testers: Get Out of the Quality Assurance Business by Michael Bolton […]

  30. Andra says:

    I’m a QA engineer, and I do not perform tests on my daily base work routine.
    Because I’m the only one in my team in the testing/QA area, I make my tests plans/scenarios/test cases etc, I also perform the audit for software, and I’m involved in the development process.
    To be honest, only 30% of my work is actually testing.
    I do not really know what area I’m located in.

    Michael replies: Well, it seems to me that you know what area you’re located in; you say yourself that you’re in the testing/QA area. In fact I disagree with some part of that, but it’s up to you, not me, to decide your roles. Reflecting on that issue, as you’re doing, is a fine start. Notice that I say “roles”, there; roles can be picked up and dropped when it suits you.

  31. […] assessment and make the ultimate business decisions as it relates to the product (more about that here from Michael Bolton). This may involve the Product Owner on your team, or it may involve a set of stakeholders who are […]

  32. Thomas says:

    Interesting and great post Michael. I’m head of QA at our company. My boss and I have had discussions about quality responsibility. He claims that I am responsible for our quality and I’m not trying to dodge or avoid taking responsibility but I surely have to question it when reading your post. As you write in the beginning; the testers (my staff) can’t change the code etc.

    Michael replies: Were I in the same room with you, I’d ask you and your boss what you think “responsibility” means. I think it would be a good idea for you to have that conversation without me, too.

    If responsibility doesn’t come with corresponding authority to make decisions, set policies, change things, it’s not really responsibility. Literally, you don’t have the authority to respond such that things change.

    I know you also wrote: Cem Kaner said “the quality assurance role in the company, lay with the management and the CEO (the principal quality officer in the company), since it was they—and certainly not the testers—who had the authority to make decisions about quality.”

    So the question for me is. If I’m not responsible for our quality. Then what am I responsible for as head of QA?

    Only you and your boss can decide that. It’s not reasonable for him to make you responsible for quality assurance. It might quite be reasonable for him to make you responsible for quality awareness. In this case, your job would be to

    • gather relevant information about the project, including an understanding of the product, the customers, the tasks or purposes that they’re attempting to fulfill while using the product;
    • sift and sort that information—and since some of that information will be missing, develop the information that’s not yet available;
    • test the product, exploring and experimenting with it, actively looking for problems that might threaten its value;
    • report on the state of the product and of the project;
    • tell a story about problems that you’ve observed in the product (bugs), and potential problems that you haven’t observed yet but that you can infer might be there (risks);
    • talk about your coverage—the testing you’ve done so far—how you have configured, operated, observed the product;
    • talk about what you haven’t covered—the testing you could do that you haven’t done yet (or will not do at all, unless your client objects);/li>
    • talk about the things in the product and the project that are making testing more difficult, harder or slower.

    If there’s a bug in the product, the information that testing reveals can help in getting the bug fixed. If there are problems in the project, that information can help to bring those problems to the attention of management—and perhaps help to change the conditions that help to breed bugs.

    Sorry for the overuse of the r-word 😉

    If you don’t have authority to change things, you don’t have responsibility for quality. It might, however, be that you get the blame.

  33. […] since no one person can “assure” quality. If interested, you can read more on that in, “Testers, Get Out of the Quality Assurance Business” by Michael […]

  34. […] since no one person can “assure” quality. If interested, you can read more on that in, “Testers, Get Out of the Quality Assurance Business” by Michael […]

  35. […] Testers: Get Out of the Quality Assurance Business: Michael Bolton (@michaelbolton), co-author of Rapid Software Testing and blogger at www.developsense.com, tells QA Testers to move on to next stage. […]

  36. […] wisdom in his presentation and it stressed on the importance of communication, Testing vs Checking, Testers getting out of the QA business and so on. Next up, was the talk from Michelle Cross on “Transformation of a QA Department”. […]

  37. […] I then moved into the paradigm of ‘It is my job to find defects and report on the status of those defects but I still know better on what should be fixed since I am the tester’ and while I explored, testing outside of the acceptance criteria, I was doing ad-hoc/random testing rather than truly structured exploratory testing. I was still missing the boat on the true nature of what it meant to do exploratory testing and I wasn’t even thinking yet on reporting out on what I did not test, only what I did. I was getting to a point where I realized there were more important product risks, but still somewhat ignoring them because I had not fully broken out of the paradigm of the tester being the final “gatekeeper” of the product/release, which we are not. (Read more on that in Michael Bolton’s blog “Testers: Get Out of the Quality Assurance Business“) […]

  38. Den says:

    Hi Michael, I read with interest your topic of Quality Assurance.

    As a tester I pride myself on being Quality Assurance. Part of my job is also to perform analysis on testing pass rates, number of bugs, where they cluster in the code and who coded them etc in order to determine why developers are making mistakes and how things can be improved to stop them making less mistakes.

    My analysis and subsequent chats with developers has uncovered 3 recurring issues that cause developers to make mistakes:

    1- Poorly communicated requirements from BA’s
    2- New developers on the project not having context of what they are building
    3- Developer fatigue

    I then discuss with the whole software team to refine processes to lessen the chance of developers making mistakes.

    It is only a tester that has an overall view of the project in this way because bugs & test cases are the artifacts we deal with all day with granularity of bugs as to who, when, were, how many times. Developers do not have time or interest to make these correlations and neither to ba’s or project managers.

    So although you say QA does not have the ability to change the code directly my view is that it is usually external factors that cause the mistakes in the code in the first place and a mindful QA can manipulate these external factors through research and understanding to get the code into a better state.

    By broaching the subject of software quality with developers in this way I have gained their respect because they know I am looking out for them.

    Kind Regards

    This does not sound like assurance to me. They do sound like worthy things to do, powerful forms of advocacy and assistance. However, it is the developers who make the changes, and the managers who run the project. Unless they take action, the quality of the product doesn’t change, and the project environment doesn’t change.

    I think it’s important that testers speak precisely about the extents and limits of what they can do. Without authority, you can’t assure quality—but that’s perfectly okay, in my view. Assistance can be valuable, and it your case, it sounds like it is. Keep it up.

    P.S. Be careful about the validity of pass rates.

  39. […] various discussions about it and I mostly lean towards the point of Michael Bolton in his post Testers: Get Out of the Quality Assurance Business. It was an eye opener blog post for me: my first job as a tester even had it in the title […]

  40. […] on several roles beyond testing. You could say that testers in this school are getting themselves into the quality assurance business. Definitely some food for thought here, and something I expect to […]

Leave a Reply

Your email address will not be published. Required fields are marked *