Correcting User Decisions Based on Incorrect Machine Learning Decisions

TL;DR

This paper demonstrates that user decision accuracy can improve through interaction with machine learning models, even when the ML's accuracy is lower than that of a human expert, especially in private communication settings.

Contribution

It provides statistical evidence that private interactions with less accurate ML models can enhance user decision-making, challenging the assumption that ML must outperform humans to be beneficial.

Findings

Interaction with ML improves user accuracy in private settings.

Even less accurate ML models can positively influence decisions.

User awareness and communication context affect decision outcomes.

Abstract

. It is typically assumed that for the successful use of machine learning algorithms, these algorithms should have a higher accuracy than a human expert. Moreover, if the average accuracy of ML algorithms is lower than that of a human expert, such algorithms should not be considered and are counter-productive. However, this is not always true. We provide strong statistical evidence that shows that even if a human expert is more accurate than a machine, an interaction with such a machine is beneficial when communication with the machine is non-public. The existence of a conflict between the user and ML model, and the private nature of user-AI communication will have the effect of making the user think about their decision and hence increase overall accuracy.