Blog Posts from November, 2013

Very Short Blog Posts (9): “Insufficient Requirements”

Wednesday, November 27th, 2013

Some people say they “don’t have enough requirements to start testing,” or that the requirements are unclear or incomplete or contradictory or out of date.

First, those people usually mean requirements documents; don’t confuse that with requirements, which may be explicit or tacit. There are plenty of sources of requirements-related information, and it’s a tester’s job to discover them and make inferences about them.

Second, insufficient clarity about requirements is both a test result and a project risk, and awareness of them may be crucially important project information. (If the testers aren’t clear on the requirements, are the programmers? And if they’re not, how is it that they’re building the product?)

Finally, if there is uncertainty about requirements, one great way around that problem is to start testing and reporting on what you find. “Insufficient requirements” may be a problem for testing—but it’s also precisely a problem that testing can help to solve.

Can You Hear The Alarm Bells?

Monday, November 25th, 2013

Many people seem certain about what happened to cause the healthcare.gov fiasco. Stories are starting to trickle out, and eventually they’ll be an ocean of them. To anyone familiar with software development, especially in large organizations, these stories include familiar elements of character and plot. From those, it’s easy to extrapolate and fill in the details based on imagination and experience. We all know what happened.

Well, we don’t. In a project of that size, no one knows what happened. No one can know what happened. Imagine Rashomon scaled up to hundreds of people, each making his own observations and decisions along the way.

As time goes by, I anticipate some people saying that the project will represent a turning point in software development and project management. “Surely,” they will say, “after a project failure of this size and scope, people will finally learn.” Alas, I’m less optimistic. As the first three premises of rapid software testing describe it, software development is a human activity that is surrounded by 1) confusion, 2) complexity, 3) volatility, 4) urgency and… 5) ambition. Increasing ambition causes increases in the other four items too. In our societies, we could help to defend ourselves against future fiascos by restraining our ambitions, but I fear that people will put blindfolds on each other, pass around the keys, and scramble to get back into the driver’s seat of the school bus. How will they do this?

One form of the blindfold is to say “That not going to be a problem here because…”

…failure is not an option.
…we have our best people on it.
…we can’t disappoint the client.
…it doesn’t have to be perfect. (thanks, Joe Miller, @lilshieste)
…we’ll fix it in production.
…no user would ever do that.
…the users will figure it out.
…the users will never notice that.
…THAT bug is in someone else’s code.
…we don’t have to fix that; that’s a new feature request.
…it’s working exactly as designed.
…if there’s no test case for it, it’s not a bug.
…the clients will come to their senses before the ship date.
…we have thousands of automated tests that we run on every build
…this time it will be different.
…we have budget to fix that before we deploy.
…at least the back end is working right.
…if there are performance problems, we’ll just add another few servers.
…we’ve done lots of projects just like this one.
…foreign-language support is something we could cut.
…that list there says that this is a level three threat, not a level one threat.
…the support people can handle whatever problems come up.
…this graph shows that the load will never get that high.
…now is too soon; we’ll tell the clients about the problems after we’ve fixed them.
…we’re thinking positively&mdashthat can-do spirit will see us through.
…we still have plenty of time left to fix that.
…the spec didn’t say anything about having to handle special characters. How are single quotes a big deal?
…the client should have thought of that before.
…seriously, that’s just a cosmetic problem.
…it’s important not to complicate things.
…everybody WILL put in some overtime and we WILL get this thing done.
…well, at least the front end looks good, and people will be happy with that.
…everyone here is committed to making sure this ships on time.
…we’ll just shorten the test cycle.
…if there’s a problem, the other/upstream/downstream team will let us know.
…they can take care of that in training.
…we’ve planned to make sure that nothing unexpected happens.
…we’ve got this fantastic new framework that’ll make things go faster.
…we’ll pull a bunch of people off other projects to work on this one.

I wonder whether these things were said, at one time or another, during the healthcare.gov project. I don’t know if they were. I don’t know what happened. I didn’t work on it. But I’ve heard these things on projects before, I know that I can listen for them, and I know that they’re a sign of trouble ahead. Are they being said on your project?

Very Short Blog Posts (8): Two Big Questions

Thursday, November 14th, 2013

Too often testing is focused on getting the right answers, rather than asking worthwhile questions and helping to get them answered. There are at least two overarching questions that a tester must ask. While looking directly at the product, at its customers, at the project, at the business, and at the relationships between them, the tester’s first question is “Is there a problem here?” When the tester observes anything looks like it might be a problem, the tester’s next question is for the testing client and for the rest of the project community: “Are we okay with this?

Very Short Blog Posts (7): Planning vs. Preparation

Sunday, November 3rd, 2013

Imagine a software project. Imagine the things that you want to accomplish, the problems you might encounter, the workarounds you could apply, the accidents (both happy and sad) that might happen, the missteps you may take, the steps you can take to prevent them; all of the actions you can perform to manage the project. Now, make a detailed plan that takes all of your expectations into account.

The more detailed your plan, the more likely it will differ from reality in important respects. Unexpected things will happen, some positive, some negative, and many of them out of your control. You can’t predict future events reliably, but you can prepare to respond to them. Therefore: you might want to relax your effort on specific plans somewhat, and emphasize developing skills and resources that will help you to deal capably with surprises.