Used Car Reviews – What Can You Learn?
One question we’re often asked is which method is best: usability testing or expert usability reviews? Well, if they were sports cars, expert usability reviews might be a Porsche (pretty decent car and better than no car at all), but usability testing would be in a different league, namely Formula 1.
So, what’s the difference?
– An expert usability review is when a usability Top Gear specialist inspects a website to identify potential usability problems
– Usability testing involves getting people from the target audience to evaluate your site whilst performing tasks
They’re quite similar in many ways in that both:
– Find and prioritise usability problems
– Evaluate designs in the context of tasks
Do they find the same problems?
The answer depends on how precisely you measure this, on balance no, not reliably. Generally speaking, expert usability reviews tend to find high level breaches of design rules and consistency. Usability testing is better at finding issues related to special domain knowledge and task flows.
If you were to carry out an expert usability review before conducting usability testing, and compare the two sets of findings, you can see how the issues you found compared to their real experiences. Did you find all the problems? Which ones did you miss? Which ones that you thought might be issues turned out not to be? This will give you a good feel for how expert usability reviews hold up against usability testing.
Often, expert reviews will:
– Miss usability issues that arise during usability testing
– Find some issues that usability testing didn’t
– Report false alarms (i.e. not real issues)
And that’s the problem. No matter how good an expert you are if you just rely on an expert usability review you’ll find different sorts of issues and overlook some potentially serious ones.
Consider the case of a health website aimed specifically at new mums. The expert usability review found plenty of great issues, but might have missed the main killer one that caused participants to pass comments like “That’s just so insensitive!” and “I wouldn’t use this site anymore”. And all because of a seemingly benign bit of banner advertising for a local gym placed next to an article about breast feeding. A reviewer would have difficulty predicting the indignant reaction of new mothers who felt the site was rubbing their nose in it. After all there would be little chance of them going anywhere near a gym with a newborn.