BLUEmed: Retrieval-Augmented Multi-Agent Debate for Clinical Error Detection

TL;DR

BLUEmed is a multi-agent debate framework with retrieval-augmented generation that improves clinical error detection in medical notes by combining evidence-grounded reasoning and multi-perspective verification.

Contribution

It introduces a novel multi-agent debate system with hybrid retrieval-augmented generation for enhanced clinical error detection, outperforming existing baselines.

Findings

Achieves 69.13% accuracy in clinical terminology substitution detection.

Outperforms single-agent RAG and debate-only baselines in experiments.

Retrieval augmentation and structured debate are complementary, improving detection performance.

Abstract

Terminology substitution errors in clinical notes, where one medical term is replaced by a linguistically valid but clinically different term, pose a persistent challenge for automated error detection in healthcare. We introduce BLUEmed, a multi-agent debate framework augmented with hybrid Retrieval-Augmented Generation (RAG) that combines evidence-grounded reasoning with multi-perspective verification for clinical error detection. BLUEmed decomposes each clinical note into focused sub-queries, retrieves source-partitioned evidence through dense, sparse, and online retrieval, and assigns two domain expert agents distinct knowledge bases to produce independent analyses; when the experts disagree, a structured counter-argumentation round and cross-source adjudication resolve the conflict, followed by a cascading safety layer that filters common false-positive patterns. We evaluate BLUEmed on a clinical terminology substitution detection benchmark under both zero-shot and few-shot prompting with multiple backbone models spanning proprietary and open-source families. Experimental results show that BLUEmed achieves the best accuracy (69.13%), ROC-AUC (74.45%), and PR-AUC (72.44%) under few-shot prompting, outperforming both single-agent RAG and debate-only baselines. Further analyses across six backbone models and two prompting strategies confirm that retrieval augmentation and structured debate are complementary, and that the framework benefits most from models with sufficient instruction-following and clinical language understanding.