How Accurate Does It Feel? -- Human Perception of Different Types of Classification Mistakes

TL;DR

This study explores how different types of classification mistakes influence human perception of a classifier's accuracy, revealing that perceived accuracy is affected by mistake difficulty and diverges from actual accuracy.

Contribution

It demonstrates that human perception of classifier performance varies with mistake difficulty and challenges the use of traditional accuracy metrics for user perception assessment.

Findings

Perceived accuracy varies with mistake difficulty

Actual accuracy differs from perceived accuracy

Not all mistakes impact perception equally

Abstract

Supervised machine learning utilizes large datasets, often with ground truth labels annotated by humans. While some data points are easy to classify, others are hard to classify, which reduces the inter-annotator agreement. This causes noise for the classifier and might affect the user's perception of the classifier's performance. In our research, we investigated whether the classification difficulty of a data point influences how strongly a prediction mistake reduces the "perceived accuracy". In an experimental online study, 225 participants interacted with three fictive classifiers with equal accuracy (73%). The classifiers made prediction mistakes on three different types of data points (easy, difficult, impossible). After the interaction, participants judged the classifier's accuracy. We found that not all prediction mistakes reduced the perceived accuracy equally. Furthermore, the perceived accuracy differed significantly from the calculated accuracy. To conclude, accuracy and related measures seem unsuitable to represent how users perceive the performance of classifiers.