Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance

TL;DR

This paper introduces trust-adaptive interventions in AI systems to reduce inappropriate reliance by dynamically adjusting explanations and pauses based on user trust levels, improving decision accuracy.

Contribution

It proposes a novel trust-adaptive approach that adjusts AI behavior to mitigate trust biases, demonstrated through experiments in science and medical decision-making.

Findings

Up to 38% reduction in inappropriate reliance

20% improvement in decision accuracy

Effective use of explanations and pauses for trust calibration

Abstract

Trust biases how users rely on AI recommendations in AI-assisted decision-making tasks, with low and high levels of trust resulting in increased under- and over-reliance, respectively. We propose that AI assistants should adapt their behavior through trust-adaptive interventions to mitigate such inappropriate reliance. For instance, when user trust is low, providing an explanation can elicit more careful consideration of the assistant's advice by the user. In two decision-making scenarios -- laypeople answering science questions and doctors making medical diagnoses -- we find that providing supporting and counter-explanations during moments of low and high trust, respectively, yields up to 38% reduction in inappropriate reliance and 20% improvement in decision accuracy. We are similarly able to reduce over-reliance by adaptively inserting forced pauses to promote deliberation. Our results highlight how AI adaptation to user trust facilitates appropriate reliance, presenting exciting avenues for improving human-AI collaboration.