Development and Comparative Evaluation of Three Artificial Intelligence Models (NLP, LLM, JEPA) for Predicting Triage in Emergency Departments: A 7-Month Retrospective Proof-of-Concept

TL;DR

This study compares three AI models for emergency triage prediction, finding that an LLM-based model outperforms others and nurses, suggesting potential for improving safety and efficiency in emergency departments.

Contribution

It introduces and evaluates three AI models, highlighting the superior performance of an LLM-based approach in emergency triage prediction.

Findings

LLM-based URGENTIAPARSE achieved highest accuracy (F1 0.900)

LLM model showed robustness with raw and structured data

AI integration could improve safety and efficiency in EDs

Abstract

Emergency departments struggle with persistent triage errors, especially undertriage and overtriage, which are aggravated by growing patient volumes and staff shortages. This study evaluated three AI models [TRIAGEMASTER (NLP), URGENTIAPARSE (LLM), and EMERGINET (JEPA)] against the FRENCH triage scale and nurse practice, using seven months of adult triage data from Roger Salengro Hospital in Lille, France. Among the models, the LLM-based URGENTIAPARSE consistently outperformed both AI alternatives and nurse triage, achieving the highest accuracy (F1-score 0.900, AUC-ROC 0.879) and superior performance in predicting hospitalization needs (GEMSA). Its robustness across structured data and raw transcripts highlighted the advantage of LLM architectures in abstracting patient information. Overall, the findings suggest that integrating LLM-based AI into emergency department workflows could significantly enhance patient safety and operational efficiency, though successful adoption will depend on addressing limitations and ensuring ethical transparency.