MedOrch: Medical Diagnosis with Tool-Augmented Reasoning Agents for Flexible Extensibility

TL;DR

MedOrch is a modular AI framework that integrates specialized tools and reasoning agents to enhance medical diagnosis and decision support, providing transparency and adaptability in complex healthcare tasks.

Contribution

MedOrch introduces a flexible, agent-based architecture for integrating external medical tools and reasoning processes, improving adaptability and transparency in AI-driven healthcare decision-making.

Findings

Achieves 93.26% accuracy in Alzheimer's diagnosis, outperforming baselines.

Attains 50.35% accuracy in predicting disease progression.

Demonstrates superior performance in chest X-ray analysis and visual question answering.

Abstract

Healthcare decision-making represents one of the most challenging domains for Artificial Intelligence (AI), requiring the integration of diverse knowledge sources, complex reasoning, and various external analytical tools. Current AI systems often rely on either task-specific models, which offer limited adaptability, or general language models without grounding with specialized external knowledge and tools. We introduce MedOrch, a novel framework that orchestrates multiple specialized tools and reasoning agents to provide comprehensive medical decision support. MedOrch employs a modular, agent-based architecture that facilitates the flexible integration of domain-specific tools without altering the core system. Furthermore, it ensures transparent and traceable reasoning processes, enabling clinicians to meticulously verify each intermediate step underlying the system's recommendations. We evaluate MedOrch across three distinct medical applications: Alzheimer's disease diagnosis, chest X-ray interpretation, and medical visual question answering, using authentic clinical datasets. The results demonstrate MedOrch's competitive performance across these diverse medical tasks. Notably, in Alzheimer's disease diagnosis, MedOrch achieves an accuracy of 93.26%, surpassing the state-of-the-art baseline by over four percentage points. For predicting Alzheimer's disease progression, it attains a 50.35% accuracy, marking a significant improvement. In chest X-ray analysis, MedOrch exhibits superior performance with a Macro AUC of 61.2% and a Macro F1-score of 25.5%. Moreover, in complex multimodal visual question answering (Image+Table), MedOrch achieves an accuracy of 54.47%. These findings underscore MedOrch's potential to advance healthcare AI by enabling reasoning-driven tool utilization for multimodal medical data processing and supporting intricate cognitive tasks in clinical decision-making.