DevelopsenseLogo

On Testing and Checking Refined

Over the last few months, and especially during some face-to-face time that we had in England recently, James Bach and I have been working to sharpen our notions of testing and checking. Although the task had been on the list for some time, we didn’t get a sense of great urgency about it until we were surprised recently to find that, at a very subtle but important level, we meant … Read more

“Merely” Checking or “Merely” Testing

The distinction between testing vs. checking got a big boost recently from James Bach at the Øredev conference in Malmö, Sweden. But a recent tweet by Brian Marick, and a recent conversation with a colleague have highlighted an issue that I should probably address. My colleague suggested that somehow I may have underplayed the significance or importance or the worth of checking. Brian’s tweet said, “I think the trendy distinction … Read more

Testing, Checking, and Convincing the Boss to Explore

How is it useful to make the distinction between testing and checking? One colleague (let’s call him Andrew) recently found it very useful indeed. I’ve been asked not to reveal his real name or his company, but he has very generously permitted me to tell this story. He works for a large, globally distributed company, which produces goods and services in a sector not always known for its nimbleness. He’s … Read more

Context-free Questions For Testing and Checking

After a presentation on exploratory approaches and on testing vs. checking yesterday, a correspondent and old friend writes: Although the presentation made good arguments for exploratory testing, I am not sure a small QA department can spare the resources unless a majority of regression checking can be moved to automation. Particularly in situations with short QA cycles. (Notice that he and I are using “testing” and “checking” in this specific … Read more

Testing, Checking, and Changing the Language

In the course of trying to describe distinctions between testing and checking, a number of questions have come up: Do you want to change the language? Won’t saying “check” be confusing? Won’t this undermine our goal of industry-standard terminology? Won’t calling certain kinds of tests “checks” fly in the face of years of documentation and books? Isn’t this yet another case of you wanting testing to be done the same … Read more

Elements of Testing and Checking

In the last couple of weeks, I’ve been very gratified by the response to the testing-vs.-checking distinction. Thanks to all who have grabbed on to the idea and to those who have questioned it. There’s a wonderful passage in Chapter 4 of Jerry Weinberg‘s Perfect Software and Other Illusions About Testing in which he breaks down the activities of a programmer engaged in testing activities—testing for discovery, discovering an unexpected … Read more

Testing vs. Checking

Post-postscript: Think of this blog post with its feet up, enjoying a relaxing retirement after a strenuous career. Please read the new version first. In the years since the original post, I’ve further refined my take on the subject of testing and checking, mostly in collaboration with my colleague James Bach. Our current thinking on the topic appears on his blog, and I provide some followup here. We’ve also benefitted … Read more

Testing, Now More Than Ever

To all managers and executives: despite how it’s in fashion these days, it’s not a good time to be laying off testers, or to be leaving them unprepared and untrained. Software can be wonderful. It can help us with all kinds of stuff, unimaginably quickly and at enormous scale. This sounds very appealing. Skilled testers, at least, have always known that we must treat output from machinery with appropriate skepticism … Read more

A Reply to “Running a crowd-sourced experiment on using LLMs for testing”

This post and the ones that follow represent an expansion on a thread I started on LinkedIn. On September 30, 2023, Vipul Kocher — a fellow with whom I have been on friendly terms since I visited his company and his family for lunch in Delhi about 15 years ago — posted a kind of testing challenge on LinkedIn. I strongly encourage you to read the post. I’ll begin by … Read more

Response to “Testing: Bolt-on AI”

A little while back, on LinkedIn, Jason Arbon posted a long article that included a lengthy conversation he had with ChatGPT.  The teaser for the article is “A little humility and curiosity will keep you one step ahead of the competition — and the machines.”  The title of the article is “Testing: Bolt-on AI” and in Jason’s post linking to it, I’m tagged, along with my Rapid Software Testing colleague … Read more