Blog: You Won’t See It Until You Believe It

Not too long ago, I updated my copy of Quicken. I hesitate to say upgrade. I’ve been using Quicken for years, despite the fact that the user interface has never been wonderful and has consistently declined a little in each version.

One of these days, I’ll do a 90-minute session and record some observations about the product. But for now, here’s one.

The default sort order for transactions in an account listing is by date, from earliest to latest. There are options whereby you can sort by reference number, payee, the amount of money spent or received, or the category. On the right side, there’s a scroll bar. As with pretty much all scroll bars, there’s a thumb—the button-like thing that one drags on to make the scrolling happen. No matter what I’ve chose for the sorting order, the tooltip associated with the thumb stubbornly continues to display the date, and the listing doesn’t update until I have let go of the scroll bar. So the tooltip is useless, and I can’t tell how far I need to scroll.

There are a zillion little problems like that in the product that make it unnecessarily hard to use. As I’ve maintained so often before, you can’t tell from the outside that no one tested the scroll bars, but I can guarantee that no one fixed them.

Upon updating the product, I was asked to fill out a survey. Aha! A chance to provide feedback! One of the survey questions was “What was your primary reason for upgrading Quicken?”

I wanted to respond, “Anticipated bug fixes.” I wanted to respond “I was hoping against hope to see some of the user interface problems in the previous versions sorted out.” The choices that I was offered were very close to this (I didn’t record them at the time, but a later online survey offered me these choices, which are close to what I remember):

  • I received an email from Quicken/Intuit
  • My previous version was no longer supported
  • I saw it advertised
  • I wanted specific new features
  • I saw a new version in stores
  • Banker/Financial advisor recommended I upgrade
  • I read a news article that mentioned the new Quicken version

In the survey included as part of the product update, there was no “Other” with a text box to indicate why I was really updating. There was no “Other” at all. (There was an “other” option in a subsequent survey form, of which I was notified through email.) This is how marketers get to make the assertion, “No one is interested in bug fixes.” They don’t see the evidence for it. But if you systematically place blinders over your eyes, you won’t see the evidence for much of anything other than what’s right in front of you.



Marshall McLuhan is rumoured to have said, “I wouldn’t have seen it if I hadn’t believed it.” If you want to observe something, it helps to believe that it’s possible. At least, it helps not to constrain your capacity to observe something that you didn’t expect. For the same reason, test cases with pre-defined and closed outcomes intensify the risk that a tester will be blind to what’s going on around. For the same reason, certification exams that present exactly four multiple-choice answers will fail to evaluate the nuances and subtleties of what a tester might observe and evaluate.

Managers, please take note!

Want to know more? Learn about upcoming Rapid Software Testing classes here.

4 responses to “You Won’t See It Until You Believe It”

  1. Kristjan Uba says:

    Hi

    I see the small usability issue problem quite often too. Project concentrates on fixing big problems and introducing new features. But neglect the small things due to not getting feedback about them – testers often do not mention them as they are already reported.
    But we need to point out that 10 small usability problems can have the same devastating effect on the sales as one missing feature.

    The issue of not wanting to see the feedback, relates to the testers not wanting to see errors. If you don’t look for errors you will not find them.

  2. Rasmus says:

    Isn’t this how software development is done? At the start of the project you set a release criteria(0 critical, 10 high severity and 100 cosmetic bugs), midway through the project with hundreds of critical and high severity bugs you tell testers to stop reporting cosmetic bugs and “Won’t fix” the ones already reported.
    Michael, you should try emailing them, tell them that the little annoyances detract from user experience and cost you time(=money). And if they were to take inspiration from Ubuntu’s One Hundred Paper Cuts Project you would be happy to upgrade to the next version they day it was released.
    In the always blame the tester spirit – Quicken/Intuit testers should have made a solid business case for finding and fixing these bugs. Guess they skipped school on Bug Advocacy day.

  3. Robert P says:

    This isn’t limited to software; it happens everywhere. I took part in a political poll once, which covered lots of questions and topics. They finally asked a question: “What is the most important issue to you today?” my answer was not one of the ones that the poll taker could check. I phrased it differently, and she still couldn’t accept it. It tunred out that they phrased the question like an open ended question, but only had a certain number of fixed answers. When I asked which *were* available, she said they couldn’t even tell me which were available could chose from!

  4. Matt says:

    The fact that the UI bug is there doesn’t overly surprise me, since they often get vetoed for bugs that are viewed as “more pressing”. I must admit, I’m surprised that they don’t accept more open-ended feedback on their survey – was there any option later on to fill in general comments?

Leave a Reply

Your email address will not be published. Required fields are marked *