Blog Posts from November, 2017

The End of Manual Testing

Wednesday, November 8th, 2017

Testers: when we speak of “manual testing”, we help to damage the craft.

That’s a strong statement, but it comes from years of experience in observing people thinking and speaking carelessly about testing. Damage arises when some people who don’t specialize in testing (and even some who do) become confused by the idea that some testing is “manual” and some testing is “automated”. They don’t realize that software development and the testing within it are design studio work, not factory work. Those people are dazzled by the speed and reliability of automation in manufacturing. Very quickly, they begin to fixate on the idea that testing can be automated. Manual bad; automated good.

Soon thereafter, testers who have strong critical thinking skills and who are good at finding problems that matter have a hard time finding jobs. Testers with only modest programming skills and questionable analytical skills get hired instead, and spend months writing programs that get the machine to press its own buttons. The goal becomes making the automated checks run smoothly, rather than finding problems that matter to people. Difficulties in getting the machine to operate the product take time away from interaction with and observation of the product. As a result, we get products that may or may not be thoroughly checked, but that have problems that diminish or even destroy value.

(Don’t believe me? Here’s an account of testing from the LinkedIn engineering blog, titled “How We Make Our UI Tests Stable“. It’s wonderful that LinkedIn’s UI tests (checks, really) are stable. Has anyone inside LinkedIn noticed that LinkedIn’s user interface is a hot, confusing, frustrating, unusable mess? That LinkedIn Groups have lately become well-nigh impossible to find? That LinkedIn rudely pops up a distracting screen after each time you’ve accepted a new invitation to connect, interrupting your flow, rather than waiting until you’ve finished accepting or rejecting invitations? That these problems dramatically reduce the desire of people to engage with LinkedIn and see the ads on it?)

Listen: there is no “manual testing”; there is testing. There are no “manual testers”; there are testers. Checking—an element of testing, a tactic of testing—can be automated, just as spell checking can be automated. A good editor uses the spelling checker, while carefully monitoring and systematically distrusting it. We do not call spell checking “automated editing”, nor do we speak of “manual editors” and “automated editors”. Editors, just “editors”, use tools.

All doctors use tools. Some specialists use or design very sophisticated tools. No one refers to those who don’t as “manual doctors”. No one speaks of “manual researchers”, “manual newspaper reporters”, “manual designers”, “manual programmers”, “manual managers”. They all do brain- and human-centred work, and they all use tools.

Here are seven kinds of testers. The developer tests as part of coding the product, and the good ones build testability into the product, too. The technical tester builds tools for herself or for others, uses tools, and in general thinks of her testing in terms of code and technology. The adminstrative tester focuses on tasks, agreements, communication, and getting the job done. The analytical tester develops models, considers statistics, creates diagrams, uses math, and applies these approaches to guide her exploration of the product. The social tester enlists the aid of other people (including developers) and helps organize them to cover the product with testing. The empathic tester immerses himself in the world of the product and the way people use it. The user expert comes at testing from the outside, typically as a supporting tester aiding responsible testers.

Every tester interacts with the product by various means, perhaps directly and indirectly, maybe at high levels or low levels, possibly naturalistically or artificially. Some testers are, justifiably, very enthusiastic about using tools. Some testers who specialize in applying and developing specialized tools could afford to develop more critical thinking and analytical skill. Correspondingly, some testers who focus on analysis or user experience or domain knowledge seem to be intimidated by technology. It might help everyone if they could become more familiar and more comfortable with tools.

Nonetheless, referring to any of the testing skill sets, mindsets, and approaches as “manual” spectacularly misses the mark, and suggests that we’re confused about the key body part for testing: it’s the brain, rather than the hands. Yet testers commonly refer to “manual testing” without so much as a funny look from anyone. Would a truly professional community play along, or would it do something to stop that?

On top of all this, the “manual tester” trope leads to banal, trivial, clickbait articles about whether “manual testing” has a future. I can tell you: “manual testing” has no future. It doesn’t have a past or a present, either. That’s because there is no manual testing. There is testing.

Instead of focusing on the skills that excellent testing requires, those silly articles provide shallow advice like “learn to program” or “learn Selenium”. (I wonder: are these articles being written by manual writers or automated writers?) Learning to program is a good thing, generally. Learning Selenium might be a good thing too, in context. Thank you for the suggestions. Let’s move on. How about we study how to model and analyze risk? More focus on systems thinking? How about more talk about describing more kinds of coverage than code coverage? What about other clever uses for tools, besides for automated checks?

(Some might reply “Well, wait a second. I use the term ‘manual testing’ in my context, and everybody in my group knows what I mean. I don’t have a problem with saying ‘manual testing’.” If it’s not a problem for you, I’m glad. I’m not addressing you, or your context. Note, however, that your reply is equivalent to saying “it works on my machine.”)

Our most important goal as testers, typically, is to learn about problems that threaten value in our products, so that our clients can deal with those problems effectively. Neither our testing clients nor people who use software divide the problems they experience into “manual bugs” and “automated bugs”. So let’s recognize and admire technical testers, testing toolsmiths and the other specialists in our craft. Let us not dismiss them as “manual testers”. Let’s put an end to “manual testing”.