Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care

TL;DR

This study demonstrates that interactive LLMs, used as diagnostic aids in emergency medicine, significantly improve physicians' diagnostic accuracy through iterative querying and dialogue strategies.

Contribution

It introduces MedSyn, a system enabling physicians to interactively query LLMs with full clinical records, showing improved diagnostic performance in real-time clinical scenarios.

Findings

Residents' correct diagnoses increased from 58.9% to 73.4%.

Automated metrics showed significant accuracy improvements (p < 0.0001).

Dialogue strategies varied by expertise, with increased cross-expertise concordance.

Abstract

Clinical decision-making in emergency medicine demands rapid, accurate diagnoses under uncertainty. Despite benchmark progress, evidence for LLMs as interactive aids in live physician workflows remains sparse. MedSyn lets physicians iteratively query an LLM provided with the full clinical record while initially viewing only the chief complaint. Seven physicians (three seniors, four residents) completed baseline and AI-assisted sessions across 52 MIMIC-IV cases stratified by difficulty. Blinded evaluation showed residents' Hard-case correctness rose from 0.589 to 0.734; difficulty-standardised completely-correct rates confirmed a medium effect ({\Delta} = 0.092; p = 0.071; d = 0.47). Automated metrics corroborated these gains: standardised any-match accuracy improved by 0.156 (p < 0.0001), and residents showed the largest F1 gain ({\Delta} = 0.138; p < 0.0001). Dialogue analysis revealed expertise-dependent strategies (seniors asked targeted, hypothesis-driven questions; residents relied on broader queries) and cross-expertise concordance increased ({\Delta} = 0.145; p < 0.0001). Interactive LLM support meaningfully enhances diagnostic reasoning.