Blog Posts from November, 2006

Keep ’em together

Tuesday, November 28th, 2006

Today I found a very strange remark in an article called “Best Practices for Outsourcing Software Testing and Development“.

The question from the interview is What are some of the mistakes that companies commonly make when outsourcing software testing?” Sashi Reddi, CEO of AppLabs Technology, a Philadelphia-based company that specializes in software testing and development with an emphasis on quality assurance, says:

The biggest mistake is that many times companies outsource both testing and development to the same vendor — thus they have a fox guarding the henhouse! These two activities need to be done by independent groups. The testing group must be able to provide independent, objective feedback on the development process and output.

I find this a very peculiar point of view. As is common from some best-practice people, it’s a set of universal generalizations that in fact only apply to a very small corner of the universe. I think Mr. Reddi is not only incorrect and misguided about many of the purposes of testing, but also about the purpose and dynamics of feedback and of software development.

I don’t believe that testers aren’t guards; testing isn’t a supervisory activity. I agree with Cem Kaner‘s explanation of testing: “a process of technical, empirical investigation, done of behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek.”

If feedback is to be useful, it’s more likely to be helpful if it’s rapid. If there’s a problem, we probably want to find it and to start solving it as soon as possible. Separating testers and developers, as Mr. Reddi appears to promote, tends lengthen to the time in which a problem can be detected, in which it can be discussed, in which it can be demonstrated, in which it can be understood, and in which it can be fixed and retested. That’s not always the case, but a genearl tendency–what’s faster: making a remark to a person next to you, or writing up an email and/or a bug report? Moreover, feedback based on understanding is typically more useful than feedback based on oblivion. Knowing how the product is intended to work and how it actually works helps me to find the problems in it more quickly. Knowing the developers and working with them helps immensely in learning about the product.

In fact, if someone wants good feedback from me, I generally need to be as close to them as I can get. If someone expects me to evaluate a product, my feedback will tend to have more value if I can see the product; if someone wants me to assess a process, they tend to get better results if they allow me to observe the process.

Now, if my job is to act as a tester that is trying to find fault with a product–say in a context like a contract dispute or a lawsuit–that’s a different matter. In that case, the situation is one of confrontation, and keeping testers and developers separate might be necessary. But most software development isn’t like that; in most testers and developers are on the same side, working collaboratively to develop the most useful and robust product possible. One can be cautious and skeptical while collaborating.

When I’m on a testing project, I do provide independent feedback on the development process and output. That is, I’m capable of thinking independently. My feedback is objective with respect to the developers, the project managers, and the other members of the project community–to the degree that anyone can be objective, which as any student of journalism will tell you is… well, impossible. But I don’t have to be kept sequestered from developers or anyone else to call ’em like I see ’em, and doing anything less than that sounds like incompetent testing to me.

Imagine someone saying, “Sales and marketing have to be done by two independent groups. We want the marketers to be kept as far away from the salespeople as possible; we don’t want the independence and objectivity of the marketing effort to be compromised by the salespeople.”

Heuristic Approaches Everywhere

Monday, November 20th, 2006

Recently I went to parent-teacher night at my 10-year-old stepson’s school. Above the door to his classroom is a list of heuristics for solving problems that I think is just dandy for testers. They’re not called heuristics there, but that’s what they are.

  • Use logical reasoning
  • Work backwards
  • Make a picture or diagram
  • Use or look for a pattern
  • Make it simpler
  • Guess and check
  • Use or make a table
  • Act out or use objects
  • Make an organized list
  • Brainstorm

A Rapid Testing Experience

Monday, November 6th, 2006

If you’re not already member of the Software Testing group on Yahoo, I’d recommend joining it; it’s where some of my favourite context-driven testers hang out. Better yet, join the group and contribute. The frequency of postings is relatively low, but the level of discourse is quite high.

Today–November 5, 2006–there’s a wonderful thing: an experience report from Pradeep Soundararajan, a tester in India that James Bach has been coaching remotely for the last little while. As you read, you may come to realize that Pradeep’s first language is not English–his first language appears to be testing.

—Michael B.