2-Step Agent: A Framework for the Interaction of a Decision Maker with AI Decision Support

TL;DR

This paper introduces the 2-Step Agent framework to model how AI predictions influence human decision-making and outcomes, revealing potential pitfalls like misaligned priors that can worsen results without proper training.

Contribution

The paper presents a novel Bayesian framework for analyzing the causal effects of AI decision support on human beliefs and decisions, highlighting risks and the importance of model transparency.

Findings

Misaligned priors can lead to worse outcomes with AI support

Proper documentation and training are crucial for effective AI decision support

The framework enables simulation of AI-human interaction effects

Abstract

Across a growing number of fields, human decision making is supported by predictions from AI models. However, we still lack a deep understanding of the effects of adoption of these technologies. In this paper, we introduce a general computational framework, the 2-Step Agent, which models the effects of AI-assisted decision making. Our framework uses Bayesian methods for causal inference to model 1) how a prediction on a new observation affects the beliefs of a rational Bayesian agent, and 2) how this change in beliefs affects the downstream decision and subsequent outcome. Using this framework, we show by simulations how a single misaligned prior belief can be sufficient for decision support to result in worse downstream outcomes compared to no decision support. Our results reveal several potential pitfalls of AI-driven decision support and highlight the need for thorough model documentation and proper user training.