Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisions

TL;DR

This paper introduces a framework viewing AI information as reasoning cues influencing complex decisions, demonstrated through a clinical case study with insights for designing effective AI decision support systems.

Contribution

It conceptualizes AI interfaces as collections of reasoning cues and empirically investigates their roles in high-stakes clinical decision-making.

Findings

Different reasoning cues influence decision processes distinctly.

Designs should prioritize cues for variable and discretionary tasks.

Cues should adapt to evolving decision contexts.

Abstract

Artificial intelligence (AI)-based decision support systems can be highly accurate yet still fail to support users or improve decisions. Existing theories of AI-assisted decision-making focus on calibrating reliance on AI advice, leaving it unclear how different system designs might influence the reasoning processes underneath. We address this gap by reconsidering AI interfaces as collections of intelligent reasoning cues: discrete pieces of AI information that can individually influence decision-making. We then explore the roles of eight types of reasoning cues in a high-stakes clinical decision (treating patients with sepsis in intensive care). Through contextual inquiries with six teams and a think-aloud study with 25 physicians, we find that reasoning cues have distinct patterns of influence that can directly inform design. Our results also suggest that reasoning cues should prioritize tasks with high variability and discretion, adapt to ensure compatibility with evolving decision needs, and provide complementary, rigorous insights on complex cases.