Wednesday, March 07, 2007
The White Glove Heuristic and The "Unless..." Heuristic
Part of the Rapid Software Testing philosophy involves reducing waste wherever possible. For many organizations, documentation is an area where we might want to cut the clutter. It's not that documentation is valueless, but every minute that we spend on documentation is a minute that we can't spend on any other activity. Thus the value of the documentation has to be compared not only to its own cost, but to its opportunity cost. More documentation means less testing; that might be okay, and even important, but it might not.
This leads to the White Glove Heuristic: if we have documentation somewhere in our process, such that running a white-gloved finger over it would cause the glove to pick up a bunch of dust, let's at least consider applying less work to that document, or eliminating it altogether.
In the RST class, there's often push-back to this idea. That's understandable; at one point, someone started producing the document in an attempt to solve some problem or mitigate some risk. The question then becomes, "Has the situation changed such that we no longer need that document?"--and the problem I see most often is that the question is begged.
On a recent trip to India, many of the participants in the class pushed back on the very idea of reducing documentation in any way, claiming "our project managers would never accept that."
I was curious. "Have you asked them?" The answer was, as I suspected, No. "So suppose you're producing a forty-page test report for a busy executive. What if that executive only ever reads the summary on the first page? Might she approve of a shorter document? If she had important questions about things in that document, could you answer those questions at a lower cost than preparing the big document?" Maybe, came the answer. "So: your project managers would never accept changes to your test documentation, unless they're not reading the whole thing anyway. Or they'd never accept changes unless they were aware of the amount of testing time lost to preparing the document. Or they'd never accept changes unless they had the confidence that you could give them the information they needed on demand." The class participants then began to recognize that a session-based test management approach might allow them to make their testing sufficiently accountable while satisfying the executives with more lightweight summary reports.
Later in the class, we were talking about oracles, and how slippery they can be. Oracles are heuristic; that means that they often work, but they can fail, and that we learn something either way. The class presents a list of consistency oracles (the list is now a little longer than in the linked article); for example, a product should behave in a manner consistent with its history-- unless there's a compelling reason for it to be otherwise, like a feature enhancement or a bug fix.
This led me to formulate The "Unless..." Heuristic: Take whatever statement you care to make about your product, your process, or your model, and append "unless..." to it. Then see where the rest of the sentence takes you.
This leads to the White Glove Heuristic: if we have documentation somewhere in our process, such that running a white-gloved finger over it would cause the glove to pick up a bunch of dust, let's at least consider applying less work to that document, or eliminating it altogether.
In the RST class, there's often push-back to this idea. That's understandable; at one point, someone started producing the document in an attempt to solve some problem or mitigate some risk. The question then becomes, "Has the situation changed such that we no longer need that document?"--and the problem I see most often is that the question is begged.
On a recent trip to India, many of the participants in the class pushed back on the very idea of reducing documentation in any way, claiming "our project managers would never accept that."
I was curious. "Have you asked them?" The answer was, as I suspected, No. "So suppose you're producing a forty-page test report for a busy executive. What if that executive only ever reads the summary on the first page? Might she approve of a shorter document? If she had important questions about things in that document, could you answer those questions at a lower cost than preparing the big document?" Maybe, came the answer. "So: your project managers would never accept changes to your test documentation, unless they're not reading the whole thing anyway. Or they'd never accept changes unless they were aware of the amount of testing time lost to preparing the document. Or they'd never accept changes unless they had the confidence that you could give them the information they needed on demand." The class participants then began to recognize that a session-based test management approach might allow them to make their testing sufficiently accountable while satisfying the executives with more lightweight summary reports.
Later in the class, we were talking about oracles, and how slippery they can be. Oracles are heuristic; that means that they often work, but they can fail, and that we learn something either way. The class presents a list of consistency oracles (the list is now a little longer than in the linked article); for example, a product should behave in a manner consistent with its history-- unless there's a compelling reason for it to be otherwise, like a feature enhancement or a bug fix.
This led me to formulate The "Unless..." Heuristic: Take whatever statement you care to make about your product, your process, or your model, and append "unless..." to it. Then see where the rest of the sentence takes you.
