More Than Accuracy: Towards Trustworthy Machine Learning Interfaces for Object Recognition

TL;DR

This study explores how visualization and perceived error plausibility influence user trust and reliance on ML object recognition systems, emphasizing the importance of transparent communication beyond mere accuracy metrics.

Contribution

It provides insights into user perceptions of ML visualizations, highlighting the role of error plausibility and probability in trust, which informs design of more trustworthy interfaces.

Findings

Users consider error plausibility and severity in trust assessment.

Probability visualization improves user understanding of system predictions.

Perceived plausibility affects how users judge the severity of errors.

Abstract

This paper investigates the user experience of visualizations of a machine learning (ML) system that recognizes objects in images. This is important since even good systems can fail in unexpected ways as misclassifications on photo-sharing websites showed. In our study, we exposed users with a background in ML to three visualizations of three systems with different levels of accuracy. In interviews, we explored how the visualization helped users assess the accuracy of systems in use and how the visualization and the accuracy of the system affected trust and reliance. We found that participants do not only focus on accuracy when assessing ML systems. They also take the perceived plausibility and severity of misclassification into account and prefer seeing the probability of predictions. Semantically plausible errors are judged as less severe than errors that are implausible, which means that system accuracy could be communicated through the types of errors.