Early prediction of in-hospital deterioration after emergency department admission using machine learning models

TL;DR

This study shows that machine learning models can better predict patient deterioration in hospitals than traditional methods, using data from emergency department admissions.

Contribution

A machine learning-based early warning system is developed and validated to predict in-hospital deterioration more accurately than traditional scoring systems.

Findings

XGB achieved the highest AUC of 0.87 in internal testing, outperforming traditional scores like NEWS, MEWS, and mSOFA.

The ML models maintained strong performance in external testing with an XGB AUC of 0.82, still superior to traditional methods.

Respiratory rate and oxygen support were identified as the most influential predictors of deterioration.

Abstract

Early prediction of clinical deterioration in patients admitted to general wards from the emergency department (ED) is crucial for timely interventions and improved outcomes. Traditional scoring systems often fail to account for dynamic physiological changes occurring during ED stays. Machine learning (ML) offers a promising alternative by integrating comprehensive patient data for enhanced predictive capabilities.

This study aimed to develop and validate an ML-based early warning system to predict adverse events, including cardiac arrest, mechanical ventilation, or intensive care unit (ICU) transfer, within 48 h of hospitalization.

This retrospective multicenter study included data from 169,807 patients across two medical centers to train ML models for predicting adverse events occurring within 48 h of hospitalization. The prediction time origin (T0) was the moment of hospital admission. All ED data obtained before T0 were included as model input, and adverse events were defined as those occurring within 48 h after T0. Three machine learning models were trained and evaluated: logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). The ML models were evaluated in external test cohorts with 54,515 patients from a regional teaching hospital. Model performance was assessed against traditional early warning scores, including the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS), and Modified Sequential Organ Failure Assessment Score (mSOFA) using the area under the curve (AUC).

In the internal test set, XGB achieved the highest AUC of 0.87 (95% CI: 0.85–0.89), followed by RF 0.85 (95% CI: 0.83–0.87) and LR 0.83 (95% CI: 0.81–0.85), outperforming NEWS (AUC 0.71), MEWS (0.69), and mSOFA (0.69). In the external test set, XGB maintained the best discrimination (AUC 0.82, 95% CI: 0.80–0.83), followed by RF 0.78 (95% CI: 0.77–0.80) and LR 0.77 (95% CI: 0.76–0.79), again exceeding NEWS (0.67), MEWS (0.65), and mSOFA (0.69). SHAP analyses identified respiratory rate and oxygen support as the most influential predictors.

The ML-based early warning system demonstrated strong predictive performance across diverse healthcare settings, significantly surpassing the capabilities of traditional scoring systems. This approach has the potential to improve risk stratification and enable timely interventions, ultimately enhancing patient outcomes.

The online version contains supplementary material available at 10.1186/s12873-025-01464-w.

Figures