Comments on: Comment on a Not-So-Good Article on Exploratory Testing

By: Immigration Services

Immigration Services — Fri, 20 May 2011 08:40:12 +0000

I agree with Jim . Exploratory software testing is a powerful and fun approach to testing. In some situations, it can be orders of magnitude more productive than scripted testing. I haven't found a tester yet who didn't, at least unconsciously, perform exploratory testing at one time or another. Yet few of us study this approach, and it doesn't get much respect in our field. It's high time we stop the denial, and publicly recognize the exploratory approach for what it is: scientific thinking in real-time. Friends, that's a good thing. Michael replies: Apropos of your remark, "I haven't found a tester yet who didn't, at least unconsciously, perform exploratory testing at one time or another," I now have a response to those who claim they don't do exploratory testing: Exploratory Testing is All Around You. Thank you for the comment. I wish you used your real name here, though.

By: Michael Bolton http://www.developsense.com

Michael Bolton http://www.developsense.com — Tue, 03 Nov 2009 13:06:46 +0000

@Jim…

I don't use the term "ad hoc testing". You're welcome to use it if you find it helpful.

But you're largely correct: testing is exploratory to the extent that the person doing the testing is in control of his or her process. Testing is scripted to the extent that someone else, from some time in the past, is in control of the tester's process.

Here is exploratory testing, as I understand and teach it:

http://www.developsense.com/2008/09/evolving-understanding-about.html

Cheers,

—Michael B.

By: Simon

Simon — Sat, 31 Oct 2009 16:58:55 +0000

Thanks for the list of resources. Having been testing for a couple of years myself, I have found from personal experience that scripted testing, though it has its benefits, requires a lot of effort for not much reward. Exploratory Testing seems like a much more exciting and rewarding methodology and I for one will definitely be reviewing the articles and books you have recommended.

By: Jim Knowlton

Jim Knowlton — Sat, 31 Oct 2009 01:17:43 +0000

I agree with most of what you're saying here…but don't you think there's a difference between exploratory testing (planned, scheduled, take lots of notes, defined sessions) and ad-hoc testing, which is much more unstructured? At the beginning of your critique you seem to imply that any user-driven exploration of the product is exploratory testing.

By: Joey

Joey — Fri, 30 Oct 2009 21:09:19 +0000

The article in question was published Feb. 25, 2003. That wasn't originally apparent on the article page itself, though the date does appear in search results on StickyMinds. We've since added it to the article for clarification. I'm not sure where the idea of its being a recent article comes from, and I hope the publishing date gives it some context.

That doesn't mean it was accurate in 2003, but rather that there have been six years of additional thought put into the subject.

Thanks for your response, Michael, and thanks for linking to it from the article.

When I was a small-time film critic for a community newspaper, I discovered that I really disliked the idea of offering up my thoughts on a film if I couldn't get a response from readers. Not only because publishing criticism has an inherent vanity to it, but because I thrive on the two-way communication that comes with discussing art. It sometimes felt like shouting from the mountaintops, when I really wanted to be down in the village, chatting with differently minded folk over a beer.

Lately, with things turning more sincerely from print to Web publishing, and with the prominence and promise of social networking, I'm starting to think that that idea has expanded beyond criticism to any area of writing. That is, most (if not all--at least nonfiction) writing benefits from a comment section.

So, again, thank you. The publication date may (or may not?) have confused things a little, but the fact that you took the time to respond adds a lot of value to publishing in general. (At least in the opinion of this former small-time film critic.)

By: Thomas Ponnet

Thomas Ponnet — Fri, 30 Oct 2009 17:29:27 +0000

I'd like to pick up the same point, that executing test scripts is seen as something that junior testers/inexperienced staff can do - no special skill required.
I believe that one needs to train their observational skills to become effective in testing.
But spreading the rumour that "anyone can execute test scripts" helps a huge area of testing companies who advocate to have one experienced test manager, some guys who are in the know who write the test scripts and the execution can be off-shored to save money.

You wrote “But sapience and skill are required for the design and construction of the check, and for the analysis and interpretation of the results, so the argument that scripted testing doesn't need skilled testers doesn't hold water if you want to do scripted testing well. Indeed, good testing of any kind requires skill.”

I often heard a Project/Programme Manager basically ask “How much does it cost more to test well?”. In other words, if we don’t outsource, how can you justify double the daily costs of a skilled tester?
THAT is the problem myself and I guess many other people in our profession are facing. Good article, thanks

By: Simon Morley

Simon Morley — Fri, 30 Oct 2009 10:07:29 +0000

There is a suggestion/assumption that scripted testing doesn't require any more observational skills than those specified in the script/document being followed. Couldn't be further from the truth (IMHO).

I think all testing requires what airline pilots call "situational awareness". (Blog post brewing...)

I hadn't seen this article - but I'll definitely have a look (I don't restrict my sources of learning - whether it be the good, the bad or the ugly!)