Blog Posts for the ‘Certification’ Category

Certification and Its Discontents I

Thursday, December 6th, 2007

EuroSTAR has been a wonderful conference for me. Alas, I’ve been unable to attend many of the track presentations because I’ve been, well, conferring. I’ve had some very engaging and interesting chats, and it’s been great to connect familiar names with their faces. I look forward to many more visits to Europe.

Today’s activities included a conversation, sometimes quite animated, with Stuart Reid. I’ve been surprised at the strength of my emotional reaction to the conversation. It has deepened my already subterranean feelings about the ISEB/ISTQB Foundation Level certification. I think this aspect of the certification business is particularly odious–tantamount to a tax on the vulnerable and the credulous.

If I understood Stuart correctly, I believe that his principal argument was that the ISEB/ISTQB Foundation Level certification demonstrates an engaged interest in the testing profession (uh, applying for, or holding, a testing job indicates an engaged interest in the testing profession, doesn’t it?) and thereby makes it easier for HR departments to filter applicants for testing jobs. (As usual, if I am mischaracterizing the argument, I welcome a correction, and I’ll publish it.) If the poor businesses don’t have the certification to separate the wheat from the chaff, how are they supposed to qualify the applicants when they receive hundreds of resumes?

Well… I suppose that the hiring staff could select applicants that send a resume and a covering letter that is clearly and neatly written, and

  • that states and articulates an engaged interest in the testing profession, in the applicant’s own words; or
  • that points to relevant life experience; or
  • that points to relevant work experience; or
  • that highlights relevant education; or
  • that points to seemingly irrelevant education or work or life experience, demonstrating a connection that might not be immediately obvious to the hiring manager, yet still intriguing; or
  • that points to self-study via books or online courses on testing; or
  • that provides a sample one-page test report for an open-source software product; or
  • that provides a link to the applicant’s work in the bug-reporting system for an online software product; or
  • that answers a set of three or four questions about testing basics provided in the advertisement; or
  • that provides direct statements of endorsement from a personal reference; or
  • that provides acknowledgment of the requirement for certification along with an argument as to why the hiring manager should consider the candidate anyway, despite the absence of the certification; or
  • that places the letters “ISEB” after the candidate’s name, with a footnote that says “I’m Still Employable, Buddy”;
  • or that uses any number of attention-getting devices that companies have been using to qualify entry-level applicants forever.

This would take too much work and too much time, was one reply. Too much compared to what? If we’re going to hire someone for a responsible technical position with the company, possibly for years, isn’t it worth a couple of minutes per applicant to select a set that we want to invite to an interview–where the real qualification begins? I don’t see any reason why our profession should shore up an incompetent HR department with a specious certification scheme. If the hiring manager doesn’t know how to hire or how to manage, why provide them with any help at all? (Jeez, why doesn’t the hiring manager go get a management certification?)

In addition, the resume-filtering argument ignores the fact that companies frequently hire testers from within. Testers often come from customer support (I did), from internal support, from development, from administration, or from internship. They come pre-qualified. Testers might also come through personal links and references–“I know someone who’d be great for that job.”

Have a look at this page of frequently asked questions from the Australia/New Zealand Testing Qualifications Board. Ask some more questions, less frequently asked: why is the exam a multiple choice test of forty questions for which there is only one correct answer per question? Why is a passing grade is 62.5% (25 out of 40)? Why are approximately half of the questions at the simplest level, such that getting the simplest questions correct, plus chance levels of achievement on the not-simplest questions is a passing grade? Why does it takes three weeks to mark? Why (especially considering the fee) is there no mechanism for feedback? Where have the ISEB/ISTQB and its affiliated boards managed to spend USD200/AUD300/GBP200 times 30,000 testers worldwide?

Anna Docherty gave a very good, though admittedly preliminary, presentation on some survey research that she did herself on certification (not just the foundation level) and its strengths and weaknesses in the field. The questions that she raised were pointed and pertinent, and she acknowledged that many questions remained unasked or unanswered in her research. In that sense, it was just like a good test report, and Anna immediately qualified herself under my personal certification scheme. I’ll discuss some suggestions for further investigation in a later post.

What truly astonished me today, though, was the number, immediacy, and directness of the statements, even from strong ISEB certification advocates, that the Foundation Level certification is “garbage”, “stupid”, and “useless”–those were their words. Come on guys; walk the talk. Trash it. Or make it free. Then trash it.

Why I Am Not Yet Certified — EuroSTAR Presentation

Wednesday, December 5th, 2007

Today, December 4 2007, I gave a presentation at EuroSTAR on “Why I Am Not (Yet) Certified“. James Bach was originally slated to give a different presentation with the same title, but I got the nod due to the untimely illness of James’ wife Lenore, which caused him to cancel his fall schedule (she’s much better now).

Stuart Reid, the chair of the conference, strongly supports the notion of certifications in their current forms. I disagree with that, but I have considerable respect for people who are willing to provide a platform for opposing views, and I therefore thank him for providing the opportunity to speak. I think the controversy opens up the discussion, and thereby strengthens the conference and the craft of testing.

As I said as I finished the presentation, I felt a little like Martin Luther nailing 42 PowerPoint slides to the screen. The talk was generally well received, but there were several conversations that I found rather sobering.

At least two people to whom I spoke–one a former ISEB instructor–told me that they had wanted to effect change in the multiple choice Foundation exams, but their experience was that that couldn’t happen unless the ISEB/ISTQB Syllabus were to change–and changing that proved an insurmountable obstacle for them.

Almost everyone who approached me afterwards said that they were glad that I had said the things that they had been thinking privately for several years. They tended to be enthusiastic but they also tended to check to see whether they were among friends before they spoke freely. The latter is a tendency we need to break. As it was, it felt like revolution and insurrection were in the air–but nobody was quite brave enough to speak up. I encourage people to talk about this stuff, out loud and in public. Open criticism of things that are damaging to the craft is a form of self-certification in my community.

The complacence and chill were disturbing, but once a group of people were together, the complaints started to flow. Many had taken the ISEB/ISTQB certifications. All but one found little to no value in it. They complained about the triviality and the one-and-only-one-answer nature of the Foundation Level exam. Saddest of all, they noted that in Britain and in several countries on the continent, almost all businesses that are hiring testers require applicants for entry-level jobs to have the ISEB/ISTQB certification. I’m pretty certain that this will have several nasty effects. First, it is likely to discourage people from entering the testing field the way many of our best testers have done–by accident and opportunity. In turn, this will make the profession more insular and less diverse. In turn, this will prevent new ideas from reaching the craft. This is very bad.

We’re already learning this business slowly enough. If you attend conferences–especially the major commercial ones–you’ll hear near endless repetition of the same themes: heavyweight planning and estimation for a task that should be nimble, rapid, and responsive; bloated approaches to test documentation and artifacts; relentless focus on confirmation, verification, and validation, and very little talk of investigation, exploration, and discovery. It’s narcotic–the conferences seem addicted to these talks, and they make the craft sleepy. If we’re going to repeat anything, let’s repeat Einstein’s notion that the we can’t solve problems by using the same level of thinking that we used when we created them.