Improving Emergency Department ESI Acuity Assignment Using Machine Learning and Clinical Natural Language Processing

TL;DR

This study developed a machine learning model called KATE that uses clinical natural language processing to predict emergency department triage acuity more accurately than nurses, potentially improving patient prioritization and outcomes.

Contribution

The paper introduces KATE, a novel ML model combining EHR data and NLP that significantly outperforms nurses in ESI acuity prediction.

Findings

KATE predicted ESI accurately 75.9% of the time, surpassing nurses' 59.8%.

KATE was 26.9% more accurate than average clinicians (p<0.0001).

On the ESI 2-3 boundary, KATE achieved 80% accuracy, compared to 41.4% for nurses.

Abstract

Effective triage is critical to mitigating the effect of increased volume by accurately determining patient acuity, need for resources, and establishing effective acuity-based patient prioritization. The purpose of this retrospective study was to determine whether historical EHR data can be extracted and synthesized with clinical natural language processing (C-NLP) and the latest ML algorithms (KATE) to produce highly accurate ESI predictive models. An ML model (KATE) for the triage process was developed using 166,175 patient encounters from two participating hospitals. The model was then tested against a gold set that was derived from a random sample of triage encounters at the study sites and correct acuity assignments were recorded by study clinicians using the Emergency Severity Index (ESI) standard as a guide. At the two study sites, KATE predicted accurate ESI acuity assignments 75.9% of the time, compared to nurses (59.8%) and average individual study clinicians (75.3%). KATE accuracy was 26.9% higher than the average nurse accuracy (p-value < 0.0001). On the boundary between ESI 2 and ESI 3 acuity assignments, which relates to the risk of decompensation, KATE was 93.2% higher with 80% accuracy, compared to triage nurses with 41.4% accuracy (p-value < 0.0001). KATE provides a triage acuity assignment substantially more accurate than the triage nurses in this study sample. KATE operates independently of contextual factors, unaffected by the external pressures that can cause under triage and may mitigate the racial and social biases that can negatively affect the accuracy of triage assignment. Future research should focus on the impact of KATE providing feedback to triage nurses in real time, KATEs impact on mortality and morbidity, ED throughput, resource optimization, and nursing outcomes.