An Empirical Evaluation of Predicted Outcomes as Explanations in Human-AI Decision-Making

TL;DR

This paper empirically investigates how explanations based on predicted outcomes influence human decision-making with AI, revealing increased reliance on AI and potential negative impacts on decision quality.

Contribution

It provides the first empirical evaluation of predicted outcome explanations in human-AI decision-making, highlighting their effects on reliance and decision accuracy.

Findings

Increased reliance on AI recommendations with outcome explanations.

Difficulty in distinguishing correct from incorrect AI suggestions.

Potential negative impact on decision quality.

Abstract

In this work, we empirically examine human-AI decision-making in the presence of explanations based on predicted outcomes. This type of explanation provides a human decision-maker with expected consequences for each decision alternative at inference time - where the predicted outcomes are typically measured in a problem-specific unit (e.g., profit in U.S. dollars). We conducted a pilot study in the context of peer-to-peer lending to assess the effects of providing predicted outcomes as explanations to lay study participants. Our preliminary findings suggest that people's reliance on AI recommendations increases compared to cases where no explanation or feature-based explanations are provided, especially when the AI recommendations are incorrect. This results in a hampered ability to distinguish correct from incorrect AI recommendations, which can ultimately affect decision quality in a negative way.