This happened to me again today in Quicken. It’s happened before. Worse, it’s an example of an extremely common phenomenon. My task here is to send an invoice to a particular person at a particular company. I fill in a part of a form — an address field — that appears right under the word “Invoice”. It’s not the topmost input element on the dialog, but it’s definitely the most naturalistic … Read more
When we talk about deep and shallow testing in the Rapid Software Testing namespace, some people might assume that we mean “deep testing” is good and decent and honourable, and that we mean “shallow” to be an insult. But we don’t. “Shallow” is not an insult. Depth and shallowness are ways of talking about the thoroughness of testing, but they’re not assessments of its value. The value or quality or … Read more
Many years ago, I went on a quest. “Coverage” seemed to be an important word in testing, but it began to occur to me that I had been thinking about it in a vague, hand-wavey kind of way. I sensed that I was not alone in that. I wanted to know what people meant by coverage. I wanted to know what I meant by coverage. In the Rapid Software Testing … Read more
Instead of asking “What tests should I automate?” consider asking some more pointed questions. If you really mean “how should I think about using tools in testing?”, consider reading A Context-Driven Approach to Automation in Testing, and Testing and Checking Refined. If you’re asking about the checking of output or other facts about the state of the product, keep reading. Really good fact checking benefits from taking account of your … Read more
Here are two questions that testers ask me pretty frequently: How can I show management the value of testing? How can I get more time to test? Let’s start with the second question first. Do you feel overwhelmed by the product space you’ve been assigned to cover relative to the time you’ve been given? Are you concerned that you won’t have enough time to find problems that matter? As testers, … Read more
Here is an edition of the reply I offered to a question that someone asked on Quora. Bear in mind that it might be a good idea to follow the links for context. If we do sanity testing before release, do we have to do regression testing? What if I told you Yes? What if I told you No? Some questions shouldn’t be answered. That is: some questions shouldn’t be … Read more
Summary: Loops of exploration, experimentation, studying, modeling, and learning are the essence of testing, not an add-on to it. The intersection of activity and models (such as the Heuristic Test Strategy Model) help us to perform testing while continuously developing, refining, and reviewing it. Testing is much more than writing a bunch of automated checks to confirm that the product can do something; it’s an ongoing investigation in which we … Read more
What does it mean to do deeper testing? In Rapid Software Testing, James Bach and I say: Testing is deep to the degree that it has a probability of finding rare, subtle, or hidden problems that matter. Deep testing requires substantial skill, effort, preparation, time, or tooling, and reliably and comprehensively fulfills its mission. By contrast, shallow testing does not require much skill, effort, preparation, time, or tooling, and cannot … Read more
Last week posted a blog entry on automation and coverage, in which I questioned the usefulness of trying to cover “everything” with automated tests, comparing them to the CCTV cameras that are in use all over the place, but especially in Britain. Despite the limitations of such schemes, there might also be some useful aspects. What might they be? For certain areas that we decide to cover with a camera, … Read more
If you don’t read the forums on the Software Testing Club, I’d recommend that you consider it. In my view, the STC is one of the more thoughtful venues for conversation about testing. (I’d recommend subscribing to the Software Testing mailing list, too.) A correspondent recently posted a request for help in recommending an automation approach. I answered something like what follows: Need to get a code coverage of at … Read more