Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations

TL;DR

This paper introduces a new measurement concept for appropriate reliance on AI advice, investigates how explanations influence this reliance, and provides experimental evidence on improving AI decision support effectiveness.

Contribution

It proposes the Appropriateness of Reliance (AoR) as a novel, quantifiable measure and examines the impact of explanations on reliance behavior through behavioral experiments.

Findings

Explanations significantly affect the AoR of AI advice.

The AoR is a useful two-dimensional measure for reliance behavior.

Providing explanations enhances the effectiveness of AI advice.

Abstract

AI advice is becoming increasingly popular, e.g., in investment and medical treatment decisions. As this advice is typically imperfect, decision-makers have to exert discretion as to whether actually follow that advice: they have to "appropriately" rely on correct and turn down incorrect advice. However, current research on appropriate reliance still lacks a common definition as well as an operational measurement concept. Additionally, no in-depth behavioral experiments have been conducted that help understand the factors influencing this behavior. In this paper, we propose Appropriateness of Reliance (AoR) as an underlying, quantifiable two-dimensional measurement concept. We develop a research model that analyzes the effect of providing explanations for AI advice. In an experiment with 200 participants, we demonstrate how these explanations influence the AoR, and, thus, the effectiveness of AI advice. Our work contributes fundamental concepts for the analysis of reliance behavior and the purposeful design of AI advisors.