From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Making

TL;DR

This paper introduces a comprehensive framework for evaluating human-AI decision-making readiness, emphasizing team safety and effectiveness over traditional accuracy metrics, and proposes new benchmarks for assessing collaboration quality.

Contribution

It presents a novel four-part taxonomy of evaluation metrics focused on team readiness, operationalized through interaction traces for deployment-relevant assessment.

Findings

Framework enables assessment of calibration and error recovery.

Connects metrics to human-AI onboarding lifecycle.

Supports development of standardized benchmarks.

Abstract

Artificial intelligence (AI) systems are deployed as collaborators in human decision-making. Yet, evaluation practices focus primarily on model accuracy rather than whether human-AI teams are prepared to collaborate safely and effectively. Empirical evidence shows that many failures arise from miscalibrated reliance, including overuse when AI is wrong and underuse when it is helpful. This paper proposes a measurement framework for evaluating human-AI decision-making centered on team readiness. We introduce a four part taxonomy of evaluation metrics spanning outcomes, reliance behavior, safety signals, and learning over time, and connect these metrics to the Understand-Control-Improve (U-C-I) lifecycle of human-AI onboarding and collaboration. By operationalizing evaluation through interaction traces rather than model properties or self-reported trust, our framework enables deployment-relevant assessment of calibration, error recovery, and governance. We aim to support more comparable benchmarks and cumulative research on human-AI readiness, advancing safer and more accountable human-AI collaboration.