AT-CXR: Uncertainty-Aware Agentic Triage for Chest X-rays

TL;DR

AT-CXR introduces an uncertainty-aware agentic system for chest X-ray triage that autonomously decides when to escalate, defer, or automate diagnoses, outperforming existing models in accuracy and efficiency under clinical constraints.

Contribution

This work presents the first uncertainty-aware agentic triage system for medical imaging, integrating confidence estimation with stepwise decision policies for autonomous chest X-ray analysis.

Findings

Both router designs outperform zero-shot models and supervised classifiers.

The system achieves higher accuracy and lower error rates at high coverage.

It operates with lower latency suitable for clinical settings.

Abstract

Agentic AI is advancing rapidly, yet truly autonomous medical-imaging triage, where a system decides when to stop, escalate, or defer under real constraints, remains relatively underexplored. To address this gap, we introduce AT-CXR, an uncertainty-aware agent for chest X-rays. The system estimates per-case confidence and distributional fit, then follows a stepwise policy to issue an automated decision or abstain with a suggested label for human intervention. We evaluate two router designs that share the same inputs and actions: a deterministic rule-based router and an LLM-decided router. Across five-fold evaluation on a balanced subset of NIH ChestX-ray14 dataset, both variants outperform strong zero-shot vision-language models and state-of-the-art supervised classifiers, achieving higher full-coverage accuracy and superior selective-prediction performance, evidenced by a lower area under the risk-coverage curve (AURC) and a lower error rate at high coverage, while operating with lower latency that meets practical clinical constraints. The two routers provide complementary operating points, enabling deployments to prioritize maximal throughput or maximal accuracy. Our code is available at https://github.com/XLIAaron/uncertainty-aware-cxr-agent.