Blog: Test Ideas for Documentation

Most people who bother with the matter at all would admit that the English language is in a bad way, but it is generally assumed that we cannot by conscious action do anything about it. Our civilization is decadent and our language — so the argument runs — must inevitably share in the general collapse. It follows that any struggle against the abuse of language is a sentimental archaism, like preferring candles to electric light or hansom cabs to aeroplanes. Underneath this lies the half-conscious belief that language is a natural growth and not an instrument which we shape for our own purposes.

Now, it is clear that the decline of a language must ultimately have political and economic causes: it is not due simply to the bad influence of this or that individual writer. But an effect can become a cause, reinforcing the original cause and producing the same effect in an intensified form, and so on indefinitely. A man may take to drink because he feels himself to be a failure, and then fail all the more completely because he drinks. It is rather the same thing that is happening to the English language. It becomes ugly and inaccurate because our thoughts are foolish, but the slovenliness of our language makes it easier for us to have foolish thoughts. The point is that the process is reversible. Modern English, especially written English, is full of bad habits which spread by imitation and which can be avoided if one is willing to take the necessary trouble. If one gets rid of these habits one can think more clearly, and to think clearly is a necessary first step toward political regeneration: so that the fight against bad English is not frivolous and is not the exclusive concern of professional writers.

George Orwell, Politics and the English Language

On those increasingly rare occasions when I’m at home, I’m in the city of Toronto, the province of Ontario, Canada.  At the beginning of July, Ontario will replace its provincial sales tax with the Harmonized Sales Tax (HST). The HST combines Canada’s federal Goods and Services Tax with something that looks a lot like the current provincial tax, but which, in essence, gets applied to more goods and services.  One positive aspect of the HST is that Ontario businesses won’t have to collect and submit two different taxes to two different governments.  On the negative side, we have to learn some new rules.  Here’s one: the amount of tax charged depends in part on where the service is delivered.  There are four rules associated with this, as documented in the Government of Canada’s GST Technical Bulletin “Place of supply rules for determining whether a supply is made in a province” I’ll show you just one of those rules, and its explanation, here:

Rule 1

Subject to the proposed place of supply rules for services that are explained in Parts II to XII of this section and in Sections 6 to 9, a supply of a service is made in a province if, in the ordinary course of business of the supplier, the supplier
(a) obtains only one address that is a home or a business address in Canada of the recipient, the home or business address in Canada of the recipient in the province,
(b) obtains more than one address described in paragraph (a), the address in the province described in that paragraph that is most closely connected with the supply, or
(c) in any other case, obtains an address in Canada of the recipient that is most closely connected with the supply.

That sounds a little confusing.  Let’s see what it means:

Rule 1 is based on the location of the recipient. The rule generally accomplishes this by determining the place of supply based on a particular address in Canada of the recipient that is obtained by the supplier in the ordinary course of business.

Okay…

The rule does not require a supplier to obtain an address of the recipient that the supplier does not already obtain in the ordinary course of its business. The determination of the relevant address in respect of a supply under Rule 1 is based on the facts taking into account the ordinary business practice of each supplier with respect to each supply.

So…

It should be noted that an address of the recipient obtained by a supplier will only be relevant for purposes of this rule if it is obtained in the ordinary course of the supplier’s business practices in connection with the supply. On the other hand, any address of the recipient obtained in the ordinary course of business of the supplier should be taken into consideration in applying the rule. The relevant address of the recipient also does not have to be an address obtained in respect of every supply made to the recipient for it to be considered a relevant address obtained in the ordinary course of business. An address of the recipient obtained by a supplier in the ordinary course of business could therefore include: an address of the recipient from which the supplier is hired in connection with a supply pursuant to an agreement for the supply (the “contracting address”); an address of the recipient that the supplier deals with in connection with a supply; or a billing address of the recipient in connection with a supply.

Oh, thank you.  That clears things up considerably. (And if you, dear reader, didn’t wade all the way through that mud, I promise I won’t be upset.)

Here’s a direct quote from the Rapid Software Testing course:  “When people say ‘that should be documented’, what they really mean is ‘that should be documented if and how and when it serves our purposes.’”  And we add, “Who will read it? Will they understand it? Is there a better way to communicate that information? What does documentation cost you?”

When people—managers, consultants (yes, like me), bureaucrats, process enthusiasts, methodologists, staff employees, executives, pundits, TV preachers—tell us do something, it behooves us to think of the value of (not) doing something, cost of (not) doing something, the risk of (not) doing something, and the quality of the work. Yet none of those things—especially quality, value to some person—are the same for different people. To me, the most important thing to start with is to ask the question, “Who are my clients? Who are my stakeholders?” Let’s call those people your client community.

For any piece of writing, I’d like to suggest that there’s an important stakeholder who is often overlooked: you. Your writing is a medium, as Marshall McLuhan would say. That is, your writing is an extension of you and your human capabilities, in particular speech and presence. Writing extends your physical being and what you might say were you immediately, physically present. As such, your written work a stand-in for you. It represents you, literally re-presents you at a different time, in a different place, and in a different form.

Another significant person in your client community is your direct client, the person you’re working for, the one who has commissioned the work. (That might be your employer, or it might be yourself.) Your writing is what you might say to your client, or on your client’s behalf, if you were actually in the room with your reader.

As most authorities on writiing will agree, your reader is extremely important and the ongoing focus of your work. Yet media present risk: as much as a medium extends and accelerates and intensifies some human capabilities and the roles of some senses (McLuhan might have said “heats them up”), it also diminishes the roles of others. While writing extends speech and intensifies the role of vision, writing diminishes conversation and the interaction between the speaker and the listener. Your listener can’t ask questions, and as Northrop Frye remarked, a book always says the same thing. Vision becomes hot, and sound becomes cool, requiring the reader to involve himself by constructing the sound of your voice. That calls for caution: if your writing is really going to represent you, it should sound in your reader’s head as you would want it to sound. Trouble is, you can’t ever know that. But you can test your writing, and if you discover problems, you might reveal information that would inform a decision to fix it.

So one really good way to test a piece of writing is to ask questions about it.

  • How would this sound if I were to utter it out loud?
  • Does it roll off the tongue naturally?
  • Would it sound like me as I’d like to be heard?
  • Would it sound like the sort of thing I would say?
  • Are these the words that I would choose?
  • Would I sound sloppy, or inflated, or confusing if I were to speak this sentence in a conversation?
  • Is there an obvious way to re-interpret or misinterpret this sentence?
  • What are the risks associated with misunderstanding?
  • What could I do to make my meaning more plain?

And, most importantly,

  • What problem am I trying to help my reader to solve?
  • Have I helped my reader to solve that problem?

Those questions, if we’re going to answer them well, require us to think carefully about our readers. We’re often writing for people that we haven’t met in person. We’re writing for people who have never heard us. Who might such people be? How do we imagine them? Might some readers be unfamiliar with the topic? Might some be unfamiliar with English? Might some be easily confused, or easily bored? What might we do to help them understand us? Should we provide examples? Stories? Make the message longer? Shorter?

Here’s another interesting question: what does the finished piece of writing—and the effort that it took to produce it—tell us about the subject of our writing? Might that information tell us something important about the subject? Here’s a hint: the Technical Bulletin, designed to help us interpret where a service is being performed, runs to 53 pages.

I suspect that no one—neither the writer of the passage I quoted above, nor his or her clients—asked any the questions that I’ve suggested here. Or if those questions were asked, no one cared enough about the information to do anything about it.  Still, as an optimist, I believe that the process is reversible.  If you’re feeling down, one antidote might be to read Orwell’s brilliant essay, and follow its advice.

And at last, I leave you, dear reader, with a question:  How is all this like testing and software development?

5 Responses to “Test Ideas for Documentation”

  1. Adam White says:

    Michael,

    Good post. I stopped reading the gov’t crap but did like your points on testing docs (which I’ll bullet form here)

    • How does it sound when I utter it out loud?
    • Does it roll off the tongue naturally?
    • Does it sound like me as I’d like to be heard?
    • Does it sound like the sort of thing I would say?
    • Are these the words that I would choose?
    • Would I sound sloppy, or inflated, or confusing if I were to speak this sentence in a conversation?
    • Is there an obvious re-interpretation or misinterpret this sentence?
    • What are the risks associated with misunderstanding?
    • What could I do to make my meaning more plain?

    Yes; on reflection, I’d add two more:

    • What is the problem I’m trying to help my reader to solve?
    • Does this document help to solve the problem?

    And, for readability, I’ll repost using bullets. Thanks!

    Another good way to test documentation is to have someone else read it out loud. Pay attention to the same points you listed. I also ask them for places the “flow” doesn’t work for them and places where they got lost. This method gives you a second level of feedback where you aren’t contributing your own bias.

    A couple of other tips I’d like to add
    - Are you using too many words?
    - Can you say what needs to be said with fewer words? (or are you using lots of words just to get some fictitious word count up because your high school English class teacher gave you a minimum word count to achieve on your assignments and you’ve never recovered?)

    Just for giggles I thought I would include this for you

    http://icanhascheezburger.com/2008/03/07/funny-pictures-i-question-the-general/

    Thanks, Adam!

  2. Glory Leung says:

    Pictures also say a thousand words. A good mix of words and some pictures can go a long way in having good docs.

    To Adam’s point on “(or are you using lots of words just to get some fictitious word count up because your high school English class teacher gave you a minimum word count to achieve on your assignments and you’ve never recovered?)”

    You can probably still get away with less words but a slightly larger font just to make the documentation look “longer” ;)

  3. Ola Hyltén says:

    I like this post, Michael. I’ll keep this in mind, and put it to practice in my own way, right away.

    I’m currently involved one project at a client in the medical device business, and I’m sure you are all to familiar with the regulatory demands there. I have already started to question the way things have been done so far and had set out to change it. For the better, hopefully.
    You just supplied me with some new tools and ideas to make this work. Thank you.

    Another project I’m involved with, at a slower pace, is being part of building a test team in a part of our own organization. I will put these thoughts to good use there to since I will, probably without to much competition, lay my hands on setting a standard for our internal documentation. The stuff that someone else will use to make the decisions about what to do with whatever bugs and risks are discovered. As well as the documentation that some one else, or me, will look at a year or two from when it’s written.

    Thanks for providing me with some more ideas to make this work!

    Oh, one system I’ve worked on both as a developer and a tester we have killed so much code, we’re at about 40% lines of code of when it was at its largest, that it now is real easy for anyone to work with. The system is also more usable, faster and smoother. Here to you can say the same thing, or more, with less words :)

  4. James Jahraus says:

    I really appreciate the links to sources of wisdom in your blog, like George Orwell’s essay “Politics and the English Language” and the book in your last post “Discussion of the Method”. You weave these sources of wisdom into the context of your posts, but I find them valuable as tools I can take and use on my own. I imagine you have a good library of asides: essays, articles, books, and maybe even music. I think a list of references to material like this would be a valuable addition to your site. Anyway I find your site a valuable place to go fishing for knowledge. Thanks.

  5. I really like what you’ve said here. I would like to suggest one other point. And that is the process of documenting something helps clarify things in your own mind (unless you’re in Politics in which case you probably lost the capability to think with clarity a long time back).

    What I mean is that even if no one else reads what you’ve written it will serve as a way to help the writer think through his/her ideas in a logical fashion.

    A case in point for me is the test plan. Getting people interested in a software test plan I might have written is damn difficult. So why bother? Because by the time I’ve finished writting one I’ve got a lot more straight in my own head. And when I’m talking to people I find it easier to talk with clarity.

    Just a thoughts.

Leave a Reply