Blog Posts for the ‘Done’ Category

Test Project Estimation, The Rapid Way

Thursday, January 25th, 2007

Erik Petersen (with whom I’ve shared one of the more memorable meals in my life) says, in the Software Testing Yahoo! group,

I know when I train testers, nearly all of them complain about not enough time to test, or things being hard to test. The lack of time is typically being forced into a completely unrealistic time frame to test against.

I used to have that problem. I don’t have that problem any more, because I’ve reframed it (thanks to Cem Kaner, Jerry Weinberg, and particularly James Bach for helping me to get this). It’s not my lack of time, because the time I’ve got is a given. Here’s a little sketch for you.

I’m sitting in my office. Someone, a Pointy-haired Boss (Ph.B.), barges in and says…

Ph.B.: “We’re releasing on March 15th. How long do you need to test this product?”

Me: (pause) Um… Let’s see. June 22.

Ph.B.: WHAT?! That can’t be!

Me: You had some other date in mind?

Ph.B.: Well, something a little earlier than that.

Me: Okay… How about February 19?

Ph.B.: WHAT!?! We want to release it on March 15th! Are you just going to sit on your hands for four weeks?

Me: Oh. So… how about I test until about, say, March 14.

Ph.B.: Well that’s… better…

Me: (pause) …but I won’t tell you that it’s ready to ship.

Ph.B.: How do you know already that it won’t be ready to ship?

Me: I don’t know that. That’s not what I mean; I’m sorry, I didn’t make myself clear. I mean that I won’t tell you whether it’s ready to ship.

Ph.B.: What? You won’t? Why not?!

Me: It’s not my decision to ship or not to ship. The product has to be good enough for you, not for me. I don’t have the business knowledge you have. I don’t know if the stock price depends on quarterly results, and I definitely don’t know if there are bonuses tied to this release. There are bunches of factors that determine the business decision. I can’t tell you about most of those. But I can tell you things that I think are important about the product. In particular, I can tell you about important problems.

Ph.B.: But when will you know when I can ship?

Me: Only you can know that. I can’t make your decision, but I can give you information that helps you to make it. Every day, I’ll learn more and more about the product and our understanding of it, and I’ll pass that on to you. I’ll focus on finding important problems quickly. If you want to know something specific about the product, I’ll run tests to find it out, and I’ll tell you about what I find. Any time you want to ask me to report my status, I’ll do that. If at any time you decide to change the ship date, I’ll abide by that; you can release before or after or on the 15th—whenever you decide that you don’t have any more important questions about the product, and that you’re happy with the answers you’ve got.

Ph.B.: So when will you have run all the tests?

Me: All the tests that I can think of? I can always think of more questions that I could ask and answer about the product—and I’ll let you know what those are. At some point, you’ll decide that you don’t need those questions answered—the questions or answers aren’t interesting enough to prevent you from releasing the product. So I’ll keep testing until I’m done.

Ph.B.: When will you be done?

Me: You’re my client; I’ll test as long as you want me to. I’ll be done when you ask me to stop testing—or when you ship.

Rapid testers are a service to the project, not an obstacle. We keep providing service until the client is satisfied. That means, for me, that there’s never “not enough time to test”; any amount of time is enough for me. The question isn’t whether the tester has enough time; the question is whether the client has enough information—and the client gets to decide that.