MedOrchestra: A Hybrid Cloud-Local LLM Approach for Clinical Data Interpretation

TL;DR

MedOrchestra introduces a hybrid cloud-local LLM framework for clinical data interpretation that balances privacy and complex task performance, demonstrating superior accuracy in pancreatic cancer staging from radiology reports.

Contribution

This work presents MedOrchestra, a novel hybrid approach combining cloud and local LLMs to improve clinical interpretation accuracy while preserving data privacy.

Findings

Achieves 70.21% accuracy on free-text reports, surpassing local models and clinicians.

Reaches 85.42% accuracy on structured reports, outperforming baseline methods.

Effectively decomposes complex clinical tasks into manageable subtasks.

Abstract

Deploying large language models (LLMs) in clinical settings faces critical trade-offs: cloud LLMs, with their extensive parameters and superior performance, pose risks to sensitive clinical data privacy, while local LLMs preserve privacy but often fail at complex clinical interpretation tasks. We propose MedOrchestra, a hybrid framework where a cloud LLM decomposes complex clinical tasks into manageable subtasks and prompt generation, while a local LLM executes these subtasks in a privacy-preserving manner. Without accessing clinical data, the cloud LLM generates and validates subtask prompts using clinical guidelines and synthetic test cases. The local LLM executes subtasks locally and synthesizes outputs generated by the cloud LLM. We evaluate MedOrchestra on pancreatic cancer staging using 100 radiology reports under NCCN guidelines. On free-text reports, MedOrchestra achieves 70.21% accuracy, outperforming local model baselines (without guideline: 48.94%, with guideline: 56.59%) and board-certified clinicians (gastroenterologists: 59.57%, surgeons: 65.96%, radiologists: 55.32%). On structured reports, MedOrchestra reaches 85.42% accuracy, showing clear superiority across all settings.