Domain-Adapted Small Language Models for Reliable Clinical Triage

TL;DR

This study demonstrates that small, open-source language models, when fine-tuned on domain-specific triage data, can reliably support clinical emergency severity assessments while preserving privacy.

Contribution

It introduces a domain-adapted small language model approach that outperforms larger models and proprietary systems in clinical triage accuracy and stability.

Findings

Qwen2.5-7B achieved the best balance of accuracy and efficiency.

Fine-tuning on pediatric triage data significantly reduced errors.

Domain-specific models outperformed GPT-4o in clinical triage tasks.

Abstract

Accurate and consistent Emergency Severity Index (ESI) assignment remains a persistent challenge in emergency departments, where highly variable free-text triage documentation contributes to mistriage and workflow inefficiencies. This study evaluates whether open-source small language models (SLMs) can serve as reliable, privacy-preserving decision-support tools for clinical triage. We systematically compared multiple SLMs across diverse prompting pipelines and found that clinical vignettes, concise summaries of triage narratives, yielded the most accurate predictions. The SLM, Qwen2.5-7B, demonstrated the strongest balance of accuracy, stability, and computational efficiency. Through large-scale domain adaptation using expert-curated and silver-standard pediatric triage data, fine-tuned Qwen2.5-7B models substantially reduced discordance and clinically significant errors, outperforming all baseline SLMs and advanced proprietary large language models (LLMs, e.g., GPT-4o). These findings highlight the feasibility of institution-specific SLMs for reliable, privacy-preserving ESI decision support and underscore the importance of targeted fine-tuning over more complex inference strategies.