DevelopsenseLogo

Tools

Test Tools

Selenium! Playwright! Cypress! Postman! There’s constant hubbub around these tools and others primarily focused on checking output—and now the din is getting louder about low-code or no-code tools for checking output. (Last year I reviewed a couple of those, here and here. Did these companies use their own tools to test their own tools? Either a Yes or No answer would be disturbing.) Want some heuristics for choosing tools of … Read more

That’s a Rap!

A few years ago, a friend introduced me to a recording of the mind-blowing musical Hamilton (if you haven’t seen or heard it, do yourself a favour and check it out). Our whole family was, for a time, obsessed with it. A few months after that, I was preparing for a talk at Xebia in Amsterdam. I was wondering how I could spice up the talk a little by doing … Read more

“What Tests Should I Automate?”

Instead of asking “What tests should I automate?” consider asking some more pointed questions. If you really mean “how should I think about using tools in testing?”, consider reading A Context-Driven Approach to Automation in Testing, and Testing and Checking Refined. If you’re asking about the checking of output or other facts about the state of the product, keep reading. Really good fact checking benefits from taking account of your … Read more

Experience Report: Katalon Studio

Warning: this is another long post. But hey, it’s worth it, right? Introduction This is an experience report of attempting to perform a session of sympathetic survey and sanity testing on a “test automation” tool. The work was performed in September 2021, with follow-up work November 3-4, 2021. Last time, the application under test was mabl. This time, the application is Katalon Studio. My self-assigned charter was to explore and … Read more

Worlds in Collision (keynote at ConTEST 2021)

Michael Bolton – Worlds in Collision (keynote at ConTEST 2021)

Listen to Michael Bolton at ConTEST 2021. What happens when builders and testers work together to develop GUI testing infrastructure?

In this keynote, Michael will give a preliminary experience report on observing, working, and collaborating with an expert tester (James Bach, co-author of Rapid Software Testing) — and an expert automator, Michael Mintz (author of Selenium Base) on a testing and learning mission.

The story is ongoing, and only beginning, but already we’re learning about key differences between world views and mind sets, and how we might go about resolving them. Check out online programs at Test Masters Academy: https://testmasters

What Should I Automate?

I get this question a lot: a tester who has just learned to program, or who has just learned about a new framework or tool set asks “Now that I’ve learned this, what should I automate?” Some people (mostly men, so it seems) go into hardware stores and see some fancy tool like a compound mitre saw. Unable to resist temptation, they imagine themselves building… something. So they buy the … Read more

Which Test Cases Should I Automate?

When someone asks “I have a big suite of manual tests; which tests (or worse, which test cases) should I automate?”, I often worry about several things. The first thing is that focusing on test cases is often a pretty lousy way to think about testing.  That’s because test cases are often cast in terms of following an explicit procedure in order to observe a specific result.  At best, this … Read more

s/automation/programming/

Several years ago in one of his early insightful blog posts, Pradeep Soundarajan said this: “The test doesn’t find the bug. A human finds the bug, and the test plays a role in helping the human find it.” More recently, Pradeep said this: Instead of saying, “It is programmed”, we say, “It is automated”. A world of a difference. It occurred to me instantly that it could make a world … Read more

A Context-Driven Approach to Automation in Testing

(We interrupt the previously-scheduled—and long—series on oracles for a public service announcement.) Over the last year James Bach and I have been refining our ideas about the relationships between testing and tools in Rapid Software Testing. The result is this paper. It’s not a short piece, because it’s not a light subject. Here’s the abstract: There are many wonderful ways tools can be used to help software testing. Yet, all … Read more

Oracles from the Inside Out, Part 8: Successful Stumbling

When we’re building a product, despite everyone’s good intentions, we’re never really clear about what we’re building until we try to build some of it, and then study what we’ve built. Even after that, we’re never sure, so to reduce risk, we must keep studying. For economy, let’s group the processes associated with that study—review, exploration, experimentation, modelling, checking, evaluating, among many others—and call them testing. Whether we’re testing running … Read more