Usability Inspection: Novice Crowd Inspectors versus Expert

TL;DR

This study compares novice crowd inspectors guided by an expert's heuristics with expert inspectors, finding that novices can identify similar usability issues more quickly and cost-effectively.

Contribution

It demonstrates that novice crowd inspectors guided by a single expert can match expert inspections in identifying usability issues, offering a cost-effective alternative.

Findings

Novice crowd inspectors find the same usability issues as experts.

Guided by a single expert, novices are more time-efficient.

Cost is lower for novice crowd inspections.

Abstract

Objective: This research study aims to investigate the use of novice crowd inspectors for usability inspection with respect to time spent and the cost incurred. This study compares the results of the novice crowd usability inspection guided by a single expert's heuristic usability inspection (novice crowd usability inspection henceforth) with the expert heuristic usability inspection. Background: Traditional usability evaluation methods are time consuming and expensive. Crowdsourcing has emerged as a cost effective and quick means of software usability evaluation. Method: In this regard, we designed an experiment to evaluate the usability of two websites and a web dashboard. Results: The results of the experiment show that novice crowd usability inspection guided by a single expert's heuristic usability inspection: a). Finds the same usability issues (w.r.t. content & quantity) as expert heuristic usability inspection. b). Is cost effective than expert heuristic usability inspection employing less time duration. Conclusion: Based on the findings of this research study, we can conclude that the novice crowd usability inspection guided by a single expert's heuristic usability inspection and expert heuristic usability inspection, on average, gives the same results in terms of issues identified.