Blog: Doing Development Work vs. Doing Quality Assurance

Here’s a case where a comment and question were worthy of a post of their own.  In reference to my recent post, Testers:  Get Out of the Quality Assurance Business, Selim Mia writes:

Hi Michael,

I have started following your blog just from past few days and I like to thank you for all of your thoughtful posts by which reflects your craftsmanship.

Thank you for reading, and thank you for thanking me.

I have solely agreed all of your points/advice/discussions on this post. I had many confusion about the term QA and QC since the start of my testing career and still have many confusion, i think other testers have the same. i have been working in a department called “QA” in my organization but doing mostly testing tasks as like other companies in Bangladesh. But along with testing we have also doing some of the QA tasks (i think) and below i have mentioned some of these:

  • Check-in Review: we check, each developer at-least once in a day Check-in their source code into the svn repository (source code management system) with the comment what changes he made for this particular check-in and also reviewer name who pair reviewed the code before check-in.
  • Code review: we check, is the code reviewed by the technology expert in witch technology project is developing in the regular interval (at least for the new developer’s code, code of complex functionalities, etc) and also we ensure that actions has been taken for all the review comments.
  • Audit Process Framework: we check, are all the development processes are following by the all project members except their have enough justification and approval not to follow the particular process(es).
  • Audit Bug repository: we ensure all the reported bugs have been taken into action (not a bug, assigned, WIP, fix, won’t fix).
  • Audit Document Management System: we ensure that all the updated version of all documents of the particular project are stored on the DMS.

Are not all above activities are part (of course, not all) of QA? Your kind words will be very much helpful to me.

Regards,
- Selim

What a great question! Thank you for asking.

The overarching mission for a tester, in my view, is to be of service to the project. Now, that’s not only the case for testers; I think it’s the overarching mission of anyone, everyone, on the project. We’re all in service to our paramount clients—the product owners, the business owners, the gold owners and the goal donors (as some Agile wags have said)—but we’re also in service to each other. When we’re thinking that way, the testers help the programmers by testing the product using a different skill set and mind set from the programmers; the programmers help the testers by providing a more testable product (log files, scriptable interfaces, and so on). Testers may help programmers to pinpoint the circumstances in which a bug happens; programmers help testers by providing explanations, test programs, hints on what to test. Testers learn to program; programmers learn to test. We support each other and learn from each other.

The Agile people for years have been advocating the idea of the self-organizing team. I believe in that too. That means that, in principle, anyone on the team is empowered to do whatever work needs to be done. So if a programmer takes on the tasks of setting up and configuring test environments, or if the tester is recruited to review code or models or bugs—activities that help to assure quality as a part of collaborative process, I’d say that’s cool.

The audit stuff gives me pause. Auditing, in my view, is a kind of testing role: gathering information with the intention of informing a decision. Auditors don’t set policy or enforce rules; they provide information to management. In many process-model-obsessed organizations (here in the West, at least) the role has taken on a different slant: auditors are a kind of process police. In such organizations, people rearrange and reprioritize their work not to optimize its value, but to keep the auditors happy. This is a form of goal displacement. To me, the priority should be on providing service and value to our clients, including each other.

In my view, if auditors discover some deviation from a set policy or a process model, I’d argue that the first step is to question the reasons for the deviation. Maybe someone is being sloppy; maybe someone is cutting corners; maybe someone is adding risk. But maybe someone has discovered a faster, less expensive, more efficient, more informative, more productive way of handling a task. Models always leave out something. Process models often leave out means by which we can encourage beneficial variation and change. I’ve never heard of an auditor reporting on some fabulous new problem-solving approach that someone has discovered internally. Most often, in my experience, process models leave out adaptability and people, as this remarkable TED talk describes.

It’s neither a tester’s job nor an auditor’s job, in my view, to set or enforce policy, and I think it’s politically dangerous for us to be perceived that way. As soon as we are perceived to be responsible for enforcement, we run the risk of being seen as tattletales, busybodies, quality police. In that kind of environment, information will soon start to be hidden, which undermines the task of investigating the product and identifying problems with it that threaten its value.

So, to the extent that you’re doing development work that helps to assure quality; to the extent that your teammates themselves are asking you to assist them; to the extent that you’re providing a service to them; to the extent that they appreciate what you’re doing as a service to them; and to the extent that they thank you for it, I’d say “rock on”, and congratulations.

In another forum, a correspondent suggested “Maybe it’s all down to the “overall” thing – be part of the process, not a megalomaniac who thinks he owns it.” I absolutely agree with that.  To the extent that you’re doing “quality assurance”; to the extent that your managers are requiring you to impose on your teammates (or even worse, to the extent that you’re imposing without being asked by anyone); to the extent that you’re slowing down the project or inflicting help; to the extent that the programmers see your work as enforcing the contents of a process model or policy document; to the extent that you are barely tolerated or outright resented—well, as always, that’s up to you and your organization. But it’s not the kind of work that I would condone or accept myself.

Again, thanks for writing.

6 Responses to “Doing Development Work vs. Doing Quality Assurance”

  1. Simon Morley says:

    Michael, good answer! I really appreciate the effort you put into responding to correspondents – which I’m sure is quite a demand!

    As you may have determined, responding to comments is a reward in itself.

    It prompts me to pose a related question, which I’d really value your opinion on: Software testing is sometimes perceived as QA and/or QC (quality assurance and control). The question is how did it end up like this? (If refrain from pot-shots!)

    History is a funny thing. I have some ideas, and there is some history. One correspondent (@jbailbel on Twitter) was kind enough to send me a link to Steven Rakitin’s presentation Software Quality Assurance Turns 50 (http://bit.ly/bDCC6u) which has a number of interesting facts and even more interesting references. It seems that the Atlas missile program hired an independent outside verification and validation tester completely outside of the prime contractor. Jerry Weinberg was charged with setting up the first internal independent test group by Thomas J. Watson for the Mercury space exploration program. In 1968, NATO convened the first conference on software engineering because, according to Rakitin (and, presumably McClure’s Introduction to the 1968 NATO Conference), “the computer industry at large was having a great deal of trouble producing large and complex software systems.”

    Now: I don’t have evidence, and I’m not much of a historian. But I do have experience in practicing and observing software development, so I do have theories that might explain it. I’d love it if someone had the time, diligence, and smarts to do what I can’t at present. They go like this:

    • There is a lot of fear in all domains of human endeavour, not least in software development.
    • When you’re not sure about how to do something, a few heuristics are available: do what people have talked about; do what other people have (claimed to have) done; do what the big boys do; imitate people with impressive records; do what people claim to be best practice. Cargo cult engineering is common.
    • Weinberg’s Zeroth Law of Bad Management says that when something isn’t working, do more of it. I’ll suggest a First Corollary: That when something that sounds plausibe isn’t working, it’s because we’re not doing enough of it instead of too much of it.
    • As I know from personal experience earlier in my career, no matter how much your own organization doesn’t resemble NASA, it’s hard to convince anyone—even yourself—that their practices don’t scale down.

    I don’t mean the simplistic answer: “someone copied it from production methods and it stuck” – in that case there must be a reason why it “stuck”. Or?

    Someone did copy it from production methods.

    Do you know if anyone has studied this?

    Someone may have studied it. The people and materials cited in Rakitin’s presentation look interesting. One fine day I’d like to read them for myself.

    If it is due to the simplistic reason then I’m disappointed… Expectations, expectations, expectations!!!

    If you want to find simplistic reasoning, you will rarely be disappointed.

  2. Okay, it’s Michael’s blog (soon, really soon my own will be ready)…but i hope he is allowing me to express my point of view into this question:

    You ask, how did we end up like this? Here’s my experience. basically i ended up like this because managers (and especially managers with whom i worked a lot) asked me a simple question: “is it okay enough to go live?”. When, in ignorance, i answered that question by “yes” or “no”. I gave my personal opinion about quality.

    To my managers i had become an oracle. And like all oracles(possible) a fallible one. I didn’t had all the information. I didn’t know the whole context. And i surely didn’t test every possible situation in the product (which even is impossible). However, still i defined a personal opinion about a product and ventilated it.

    I had become an oracle. The same oracles i learned to question. However, my managers didn’t acknowledged my opinion “as an oracle”. As they knew me and my professionalism for a long time they accepted my comments as factual. They valued my personal opinion over the facts about what i and they had learned about the product. They even valued it over their own opinions. I gave them easy decision making. A fallible one…

    In the following years, i realised that it’s hard to unlearn something that had become a habbit. Nowadays i try to stick to the facts and refrain myself from fallible opinions. I even explain to my managers why i do this, so he understands that even i can be wrong and he has to form his own decisions. Sometimes i still fall into the same trap, and so is he. But we are both getting there… :)

  3. Selim Mia says:

    Hi Michael,

    Thanks a lot for your kind and huge effort to respond my question. Also thanks for the wonderful TED talks video of Barry Schwartz.

    My pleasure; you’re welcome.

    With respects to your opinions, now couple of questions surrounding in my mind.

    I agree with you that “auditing is a kind of testing role: gathering information with the intention of informing a decision. Auditors don’t set policy or enforce rules; they provide information to management”. — I have thought like same before. We provided information (deviation of other people’s work) to management and management take the proper action to fix the issues and hence issues are being fixed. Here my question is, by fixing the issues which we informed are not we assuring other people’s work?

    No. We don’t fix the issues, and we don’t assure that they’ve been fixed. We do provide information to the people who decide upon what is to be fixed, and we provide information to the people who do the fixing, but we neither decide about the problems nor do we fix them.

    Some people get upset or depressed because of the perception that if you’re not fixing the problem, you’re not helping. But I disagree. Information that informs the decisions and the fixes is helpful, and it can be very valuable indeed.

    In your view, “if auditors discover some deviation from a set policy or a process model, I’d argue that the first step is to question the reasons for the deviation”. — As i was mentioned “unless their have enough justification and approval not to follow the particular process(es) for the particular project or situation”. In Barry Schwartz words “A wise person knows, when and how to make the exception to every rule – when to ignoring rules in the service of other objectives”.

    I agree with you that “maybe someone has discovered a faster, less expensive, more efficient, more informative, more productive way of handling a task. Models always leave out something. Process models often leave out means by which we can encourage beneficial variation and change”. — In Barry Schwartz words “a wise person knows, when and how to improvise – as real world problems are often ambagious and eld defined and the context is always changing” but “wisdom depends on experience, you need the time to get to know the people you serving, you need permission to allow improvise to try new things occasionally to fail and learn from your failures”.

    You mentioned “I’ve never heard of an auditor reporting on some fabulous new problem-solving approach that someone has discovered internally. Most often, in my experience, process models leave out adaptability and people” and “It’s neither a tester’s job nor an auditor’s job, in my view, to set or enforce policy, and I think it’s politically dangerous for us to be perceived that way.”

    There are many testers who are good but also there are some testers who are bad, who badly tested the application under test and badly communicate issues to the developers, they may lead bad effect to the team and to the project, they might slowing down the project. In the same way there may have some good auditing stuffs and some bad auditing stuffs.

    I’m talking about these good auditing stuffs who have moral wild to right by other people and beyond this they have moral skill to figure out what doing right means…..because practical wisdom is the combination of moral wild and moral skill, who knows how to use moral skills in pursuit of the right aims to serve to other people not to manipulate other people…who want to do the right things in the right way for the right reasons – words from the Barry Schwartz talks.

    This is a difficult area, for sure. I have ideas about what is morally right, what the right aims are, and so on. Yet, as a tester in service to my client, I don’t impose my world view on my client. My client—appropriately—makes the decisions about his or her business. In order to function effectively, I need to recognize that I’m not the owner of those decisions. I’ll certainly try to note (persuasively, at times) to my client when I think some problem or activity is likely to threaten his or her values, but I don’t want to impose my world view.

    There is a limit: if my client is doing something that I consider morally reprehensible, I quit.

    I’m talking about these good stuffs as you mentioned “to the extent that you’re doing development work that helps to assure quality; to the extent that your teammates themselves are asking you to assist them; to the extent that you’re providing a service to them; to the extent that they appreciate what you’re doing as a service to them; and to the extent that they thank you for it, I’d say ‘rock on’”.

    My question is, are not good auditing stuffs doing some QA tasks by which they are assuring quality of other’s work? Should we stop auditing?

    I’ve provided a few measures by which I believe that kind of auditing might be helpful or harmful. At that stage, the question is not really about what I think; it’s about what you think. You’re one of my clients too! I’m not going to tell you what you should do. Only you can be the judge of that. Instead, I’m going to provide you with questions or heuristics that might help you answer that question for yourself.

    Your kind words will be very much helpful to me.

    Your own decision will be more helpful—I hope!

    Regards,
    - Selim

  4. Yeah. Again its quite difficult to bypass the organization. If there are processes that are affecting the tester’s thinking too much, where he has to focus and loose time in something that normally will not do, when asked if the product is ready for live, the tester will say “Yes”. Its a little bit to get over it, its internal frustration, but can happen.

    Sebi

    Yes. This is a quote that I keep coming back to. It’s relatively recent; it was only written 49 years ago as I write this. I often quote the bit that starts “Too often…”, but the part before it is germane to your comment.

    When we have been working on a program for a long time, and if someone is pressing us for completion, we put aside our good intentions and let our judgment be swayed. So often, then, the results must provide the impartial judgment that we cannot bring ourselves to pronounce. One of the lessons to be learned from such experiences is that the sheer number of tests performed is of little significance in itself. Too often, the series of tests simply proves how good the computer is at doing the same things with different numbers. As in many instances, we are probably misled here by our experiences with people, whose inherent reliability on repetitive work is at best variable. With a computer program, however, the greater problem is to prove adaptability, something which is not trivial in human functions either. Consequently we must be sure that each test does some work not done by previous tests. To do this, we must struggle to develop a suspicious nature as well as a lively imagination.”

    —Jerry Weinberg

  5. Great post! Selim’s letter is the same as our organization’s situation.

  6. This came out while I was on holiday, and I’ve only just come across it. I found it very interesting because of my own thoughts on the relationship between testing and auditing. Maybe it’s just as well I didn’t find it before I wrote my piece on my blog.

    I agree with you about what testers and auditors /should/ be doing. In particular, I absolutely agree with you about the very real danger of goal displacement. Organisations with auditors who’re no better than half-trained compliance monkeys are sick. The goal displacement problem is widespread, and auditors should root it out and challenge it. The last thing they should do is encourage it, which many of them do when they simply report non-compliance without consideration of the context or the risks.

    It used to drive me crazy as an auditor when I caught people doing what they believed to be wrong, just because they thought it would keep us happy. The result was usually pointless, time-wasting bureaucratic controls, and it was the auditors who were being blamed for it! It was tempting to scream “don’t try and second guess me, do what’s right, then justify it”.

Leave a Reply