Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian

TL;DR

Dr. Copilot is a multi-agent LLM system designed to improve the communication quality of Romanian-speaking doctors in telemedicine by providing real-time, interpretable feedback on written responses, leading to measurable improvements in user reviews.

Contribution

This paper introduces a novel multi-agent LLM system optimized for Romanian telemedicine, focusing on enhancing communication quality rather than medical accuracy, with real-world deployment and evaluation.

Findings

Measurable improvement in response quality and user reviews

Successful deployment with 41 doctors in a real-world setting

Effective feedback on communication along 17 interpretable axes

Abstract

Text-based telemedicine has become increasingly common, yet the quality of medical advice in doctor-patient interactions is often judged more on how advice is communicated rather than its clinical accuracy. To address this, we introduce Dr. Copilot , a multi-agent large language model (LLM) system that supports Romanian-speaking doctors by evaluating and enhancing the presentation quality of their written responses. Rather than assessing medical correctness, Dr. Copilot provides feedback along 17 interpretable axes. The system comprises of three LLM agents with prompts automatically optimized via DSPy. Designed with low-resource Romanian data and deployed using open-weight models, it delivers real-time specific feedback to doctors within a telemedicine platform. Empirical evaluations and live deployment with 41 doctors show measurable improvements in user reviews and response quality, marking one of the first real-world deployments of LLMs in Romanian medical settings.