Blog: Structures of Exploratory Testing: Resources

Update, 2012-09-11: The list below has been updated over time, and now exists as part of the resources page on my Web site. Feel free to share; comments and suggestions for additions are welcome.

In a Webinar that he did for uTest on December 10, James Whittaker mused aloud about what a great idea it would be to structure exploratory testing and capture ideas about it in a repository for sharing with others. It seems to me that one ideal version of that would take the form of a bibliography in a book about exploratory testing, but apparently that’s not available. Yet I digress.

The fact is, people have been doing exactly that for years. And I do like the idea of having a repository and sharing, so here’s a survey of some exploratory testing structures and some writing about them that I hope people will find helpful. There are some excellent books out there, but for now, these ones are all online and free. Expect updates.

  • Evolving Work Products, Skills and Tactics, ET Polarities, and Test Strategy. James Bach, Jon Bach, and I authored the latest version of the Exploratory Skills and Dynamics list. This is a kind of evolving master list of exploratory testing structures. James describes it here.
  • Oracles. The HICCUPPS consistency heuristics, which James Bach initiated and which I wrote about in this article for Better Software in 2005. (Actually, at the time it was only HICCUPP—History, Image, Comparable Products, Claims, User Expectations, Purpose, Product—but since then we’ve also added S, for Standards and Statutes. Mike Kelly also talks about HICCUPP here.
  • Test Strategy. James Bach’s Heuristic Test Strategy Model isn’t restricted to exploratory approaches, but certainly helps to guide and structure them.
  • Data Type Attacks, Web Tests, Testing Wisdom, Heuristics, and Frameworks. Elisabeth Hendrickson’s Test Heuristics Cheat Sheet is a rich set of guideword heuristics and helpful reference information.
  • Context Factors, Information Objectives. Cem Kaner most recently delivered his Tutorial on Exploratory Testing for the QAI Quest Conference in Chicago, 2008. There’s a similar, but not identical talk here.
  • Quick Tests. In our Rapid Software Testing course, James Bach and I talk about quick tests. The course notes are available for free. Fire up Acrobat and search for “Quick Tests”.
  • Coverage (specific). Michael Hunter’s You Are Not Done Yet is a detailed set of coverage ideas to help prompt further exploration when you think you’re done.
  • Coverage (general). James Bach wrote this article in 2001, in which he summarizes test coverage ideas under the mnemonic “San Francisco Depot.”—Structure, Function, Data, Platform, and Operations. Several years later, I convinced him to add an element to the list, so now it’s “San Francisco Depot. The last T is for… 
  • Time. I realized a few years ago that some guideword heuristics might help us to pay attention to the ways in which products related to time, and vice versa. That turned into a Better Software article called “Time for New Test Ideas”.
  • Tours. Mike Kelly’s FCC CUTS VIDS Touring Heuristics (note the date) provides a set of structured approaches for touring the application. 
  • Stopping Heuristics. There are structures to deciding when to stop a given test, a line of investigation, or a test cycle. I catalogued them here, and Cem Kaner made a vital addition here.
  • Accountability, Reporting Progress. James and Jon Bach’s description of Session-Based Test Management is a set of structures for making exploratory testing sessions more accountable.
  • Procedure. The General Functionality and Stability Test Procedure. It was designed for Microsoft in the late 1990s by James Bach, and may be the first documented procedure to guide exploratory test execution and investigation.
  • Emotions. I gave a talk on emotions as powerful pointers to test oracles at STAR West in 2007. That helped to inspire some ideas about…
  • Noticing, Observation. At STAR East 2009, I did a keynote talk on noticing, which can be important for exploratory test execution. The talk introduces a number of areas in which we might notice, and some patterns to sharpen noticing.
  • Leadership. For the 2009 QAI Conference in Bangalore, India, I did a plenary talk in which I noted several important structural similarities between exploratory testing and leadership.

So, there it is: a repository. It now exists as part of the resources page on my Web site. Feel free to share; comments and suggestions for additions are welcome.

4 Responses to “Structures of Exploratory Testing: Resources”

  1. yvettefrancino says:

    Excellent set of resources. I've just started perusing them all, but I'm planning a more in depth study.

    I'm envisioning a Web application where types of bugs are cataloged. These could be used to help testers with planning. If test scripts were stored, there'd be some potential for reusability. As testers found more patterns of bugs these would be added.

    Do you know if any such application exists?

  2. Richard says:

    Nice list, but I must admit I took the most joy from the "Digression" as I have just read a rather mediocre book called "Exploratory testing" that fails to mention any of these sources, yet probably should have :) keep up the good work!

    Richard

  3. Alexander says:

    Hey,

    really like the source still updating it or is it outdated?

    Alex

    Michael replies: I update it from time to time. If there’s a significant update, I’ll blog it.

  4. [...] The list is designed to cover every type of consistency-expectation that testers rely on. If they realize the list is incomplete, they add a new type. [...]

Leave a Reply