Why not both? Complementing explanations with uncertainty, and the role of self-confidence in Human-AI collaboration

TL;DR

This paper empirically investigates how combining uncertainty estimates and explanations in AI models influences human trust, reliance, and understanding, especially considering users' self-confidence, to improve human-AI collaboration in critical domains.

Contribution

It provides insights into the combined effects of explanations and uncertainty on user behavior and trust, highlighting the role of self-confidence in collaborative decision-making.

Findings

Combining explanations with uncertainty estimates enhances user trust.

Self-confidence significantly influences reliance and decision-switching behaviors.

The interaction between explanations, uncertainty, and self-confidence affects human-AI collaboration outcomes.

Abstract

AI and ML models have already found many applications in critical domains, such as healthcare and criminal justice. However, fully automating such high-stakes applications can raise ethical or fairness concerns. Instead, in such cases, humans should be assisted by automated systems so that the two parties reach a joint decision, stemming out of their interaction. In this work we conduct an empirical study to identify how uncertainty estimates and model explanations affect users' reliance, understanding, and trust towards a model, looking for potential benefits of bringing the two together. Moreover, we seek to assess how users' behaviour is affected by their own self-confidence in their abilities to perform a certain task, while we also discuss how the latter may distort the outcome of an analysis based on agreement and switching percentages.