DevelopsenseLogo

Who’s to Blame?

A couple of people, both SDETs at Microsoft, have responded either to Joel Spolsky’s post on SDET culture at Microsoft or to my excerpt of it in a recent blog post of usefulness in software. Those people deserve a response.

The first issue is that they’ve taken Joel to task for promulgating rumours. But these are rumours in the same sense that the cell theory and the theory of natural selection are theories; usefully complete, good-enough summaries of the issue for the purpose of furthering the discussion–in the same sense that we spoke of Conrad Black’s alleged embezzlement. I’ve heard about the issue of SDET hiring focus from at least a dozen people, and most of them work at Microsoft.

I can’t speak for Joel, but I can speak for my own observations: I’m not blaming the SDETs for the problems in Vista. The decision to ship a product, irrespective of the shape that it’s in, is always a business decision. Testers aren’t to blame for the flaws in the product. We don’t put ’em in, and if we find them, we are to some degree lucky.

Wait–why lucky? Doesn’t skill matter? What about experience? Don’t tools help? Sure they do; all those things can help a lot. But… products and the systems that they run on are hideously complex. The set of tests that we could run is potentially infinite. Problems are unexpected, subjective, numerous, and many of them can’t be identified by a non-sapient machine. Most of all, the problems in software are usually hidden. If we’re searching for problems an intractably large space in very limited time, luck will play a part.

Testers don’t (and shouldn’t) direct product development. Program managers, project managers, product owners, senior managers–whatever they might be called–do that. Those people have authority over the budget; the schedule; the scope of the problem that it’s intended to solve; staffing, hiring and firing; business relationships; contractual obligations; annual bonuses; and the decision to ship the product. They also have control over its design, how much it’s going to pester the user, and whose software is going to ask (persistently; very persistently in Vista’s case) for you to install it. For a price.

So the SDETs aren’t responsible for the problems in Vista. What is responsible, I think, is the culture. That culture leads what I perceive to be an overly heavy automation strategy and a testing department dominated by things that programmers value, rather than things that customers value. That’s a problem, but it’s only a symptom of a wider problem. That problem results in a Vista that is so annoying that I spent a substantial amount of time and effort to make sure that it didn’t get installed on machines that I use regularly. Vista isn’t valuable to me (and, I believe, to millions of others), even though it might be functionally correct. It does perfectly all kinds of things that I don’t want it to do; and it fails to do at all things that I want it to do. That was my point, and I think it was one of Joel’s, too.

1 reply to “Who’s to Blame?”

  1. There can be many factors that lead to failure of a product. I have seen products that had high expectations set on them and suprehype created before release, trembling after release.
    One of the reasons i feel is that architechts try to fit in too many things in the product (most of the cases more that what is necessary). The problem is not with the architechture per-se, but the whole system to develop and test it (as you mentioned the culture,in a way). System is designed to be very complex (though it might be user-friendly at the end of the day),Proper testing systems are not developed around it, People are more worried about the release of the product into the market (instead of worrying about a GOOD release). In the attempt to gain a BIG victory they overlook or ignore the granular details. I’ve also seen that the managment teams (people who decided the go no-go) take testing for granted (in the same way as they take development for granted) thus making testing an eye wash process with no real benefit for the product.
    For every product in development,the life cycle,phases,processes etc must always be new. Though we can have a generic framework the actuals for the product must be carefully designed and implemented and this is where teams go wrong and finally blame it on testers.

    Reply

Leave a Comment