SOLVE-Med: Specialized Orchestration for Leading Vertical Experts across Medical Specialties

TL;DR

SOLVE-Med is a multi-agent system that uses specialized small language models and dynamic routing to improve medical question answering across multiple specialties, addressing deployment challenges like hallucinations and privacy.

Contribution

It introduces a novel multi-agent architecture with domain-specific models and a dynamic router for effective medical query answering, enabling local deployment and improved performance.

Findings

Achieved ROUGE-1 of 0.301 and BERTScore F1 of 0.697 on Italian medical forum data.

Outperformed larger standalone models up to 14B parameters.

Demonstrated effective deployment in diverse medical specialties.

Abstract

Medical question answering systems face deployment challenges including hallucinations, bias, computational demands, privacy concerns, and the need for specialized expertise across diverse domains. Here, we present SOLVE-Med, a multi-agent architecture combining domain-specialized small language models for complex medical queries. The system employs a Router Agent for dynamic specialist selection, ten specialized models (1B parameters each) fine-tuned on specific medical domains, and an Orchestrator Agent that synthesizes responses. Evaluated on Italian medical forum data across ten specialties, SOLVE-Med achieves superior performance with ROUGE-1 of 0.301 and BERTScore F1 of 0.697, outperforming standalone models up to 14B parameters while enabling local deployment. Our code is publicly available on GitHub: https://github.com/PRAISELab-PicusLab/SOLVE-Med.