CXReasonAgent: Evidence-Grounded Diagnostic Reasoning Agent for Chest X-rays

TL;DR

CXReasonAgent is a diagnostic reasoning system for chest X-rays that combines language models with clinical tools to produce evidence-grounded, reliable, and verifiable diagnostic responses, addressing limitations of existing vision-language models.

Contribution

The paper introduces CXReasonAgent, a novel system integrating LLMs with clinical diagnostic tools for evidence-grounded reasoning in chest X-ray diagnosis.

Findings

Produces faithfully grounded diagnostic responses

Enables more reliable and verifiable reasoning

Outperforms existing vision-language models in clinical fidelity

Abstract

Chest X-ray plays a central role in thoracic diagnosis, and its interpretation inherently requires multi-step, evidence-grounded reasoning. However, large vision-language models (LVLMs) often generate plausible responses that are not faithfully grounded in diagnostic evidence and provide limited visual evidence for verification, while also requiring costly retraining to support new diagnostic tasks, limiting their reliability and adaptability in clinical settings. To address these limitations, we present CXReasonAgent, a diagnostic agent that integrates a large language model (LLM) with clinically grounded diagnostic tools to perform evidence-grounded diagnostic reasoning using image-derived diagnostic and visual evidence. To evaluate these capabilities, we introduce CXReasonDial, a multi-turn dialogue benchmark with 1,946 dialogues across 12 diagnostic tasks, and show that CXReasonAgent produces faithfully grounded responses, enabling more reliable and verifiable diagnostic reasoning than LVLMs. These findings highlight the importance of integrating clinically grounded diagnostic tools, particularly in safety-critical clinical settings. The demo is available \href{https://ttumyche.github.io/cxreasonagent/#demo}{here}.