Web Analytics and Usability Testing
Presented at: 2009 UPA Conference
With Josh Ledwell (@ledwell)
Summary
Development of new Web Analytics methods has accelerated recently, and some of these methods, such as web session recording, are beginning to encroach on the territory formerly held by traditional, moderated usability testing.
Web analytics have come a long way from the log analysis techniques of a few years ago. It is now possible to see exactly where users are clicking on a page, or even to replay whole user sessions. At the same time, usability testing is becoming more of a hands-off activity, including trends toward remote and unmoderated testing. Analytics can provide data from large samples, and insight into user behavior 'in the wild.' Does this mean that we are witnessing a trend toward phasing out of usability testing as a useful technique? I propose that far from replacing usability testing, analytics can in fact help strengthen usability testing findings and recommendations, and vice versa.
Triangulation
Triangulation is the technique of using multiple methods to confirm and validate findings. Analytics and usability testing, because they do have complementary strengths, naturally lend themselves to triangulation. Usability testing using the think-aloud protocol provides rich information about user needs and intent which is missing from after-the-fact data analysis. Ambiguous findings from analytics can be clarified by a round of focused usability testing. On the other hand, the usability problems identified through testing can be further explored and confirmed using analytics. While usability testing identifies the problems, their extent and severity can be confirmed using analytics. The resulting recommendations, supported by a combination of methods, are more actionable and more likely to be taken seriously by stakeholders.