UCAgents: Unidirectional Convergence for Visual Evidence Anchored Multi-Agent Medical Decision-Making

TL;DR

UCAgents introduces a hierarchical multi-agent framework that enforces unidirectional convergence and structured evidence auditing to improve visual reasoning and accuracy in medical decision-making, reducing noise and enhancing trustworthiness.

Contribution

The paper proposes UCAgents, a novel multi-agent framework that anchors reasoning to visual evidence through unidirectional convergence and evidence auditing, addressing reasoning detachment in medical VLMs.

Findings

Achieves 71.3% accuracy on PathVQA, outperforming previous methods by 6.0%.

Reduces token cost by 87.7%, improving computational efficiency.

Balances visual evidence extraction with textual noise suppression.

Abstract

Vision-Language Models (VLMs) show promise in medical diagnosis, yet suffer from reasoning detachment, where linguistically fluent explanations drift from verifiable image evidence, undermining clinical trust. Recent multi-agent frameworks simulate Multidisciplinary Team (MDT) debates to mitigate single-model bias, but open-ended discussions amplify textual noise and computational cost while failing to anchor reasoning to visual evidence, the cornerstone of medical decision-making. We propose UCAgents, a hierarchical multi-agent framework enforcing unidirectional convergence through structured evidence auditing. Inspired by clinical workflows, UCAgents forbids position changes and limits agent interactions to targeted evidence verification, suppressing rhetorical drift while amplifying visual signal extraction. In UCAgents, a one-round inquiry discussion is introduced to uncover potential risks of visual-textual misalignment. This design jointly constrains visual ambiguity and textual noise, a dual-noise bottleneck that we formalize via information theory. Extensive experiments on four medical VQA benchmarks show UCAgents achieves superior accuracy (71.3% on PathVQA, +6.0% over state-of-the-art) with 87.7% lower token cost, the evaluation results further confirm that UCAgents strikes a balance between uncovering more visual evidence and avoiding confusing textual interference. These results demonstrate that UCAgents exhibits both diagnostic reliability and computational efficiency critical for real-world clinical deployment. Code is available at https://github.com/fqhank/UCAgents.