# Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model

## TL;DR

This study evaluated the accuracy of an updated sepsis prediction model across multiple US hospitals, finding it effective but with high variability and many false alarms.

## Contribution

The study provides the first multicenter validation of the updated Epic Sepsis Model version 2, highlighting its performance variability and alert burden.

## Key findings

- The model showed an area under the receiver operating curve between 0.82 and 0.92 across institutions.
- Positive predictive values were low, ranging from 0.13 to 0.26, indicating many false positives.
- Alert burden was high, suggesting a need for workflow adjustments to manage false alarms.

## Abstract

How accurate is the Epic Sepsis Model version 2, an updated proprietary sepsis prediction model implemented at hundreds of US hospitals, at predicting sepsis?

This prognostic study of 227 091 inpatient encounters across 4 major US health systems found that the model had an area under the receiver operating curve between 0.82 and 0.92 but demonstrated high institutional variability, low positive predictive value, and high alert burden.

These findings suggest that institutions implementing this model should conduct local validation studies to verify performance, integrate clinical workflows to manage false positives, and implement alert silencing strategies to reduce alert burden.

This prognostic study compares the performance of Epic Sepsis Model version 2 against the original Epic Sepsis Model and against clinician recognition across 4 major US health systems.

The Epic Sepsis Model version 2 (ESM v2) is a widely implemented proprietary sepsis prediction model, but no multicenter, external validation of its performance has been reported to guide adoption and use.

To conduct a multicenter validation of the ESM v2 to compare performance against the original ESM v1, outline differences across heterogenous clinical sites, and compare model performance against clinician recognition of sepsis.

This prognostic study included adult inpatient encounters at 4 large US health systems between August 31, 2023, and March 11, 2025. At each site, data were collected for a consecutive period immediately following new model implementation. Data were analyzed from July 23 to August 19, 2025.

Sepsis was defined using Sepsis-3 clinical consensus criteria. Model discrimination was assessed using area under the receiver operating characteristic curve (AUROC) at the encounter level and prediction level with 4-hour, 12-hour, and hospitalization-wide time horizons. Performance against clinician recognition of sepsis was measured using antibiotics, lactate, and body culture orders.

Of 227 091 inpatient encounters, 7401 (3.3%; median [IQR] age, 65 [54-75] years; 3359 [45.4%] female; 2.7% Asian; 24.6% Black; 64.6% White; 7.1% Hispanic ethnicity) met sepsis criteria and 219 690 (96.7%; median [IQR] age, 48 [33-65] years; 123 563 [56.2%] female; 2.5% Asian; 38.8% Black; 49.6% White; 9.6% Hispanic ethnicity) did not. At the encounter level, the AUROC ranged from 0.82 (95% CI, 0.81-0.83) to 0.92 (95% CI, 0.92-0.93) across study sites. At the prediction level with a 12-hour time horizon, the AUROC ranged from 0.75 (95% CI, 0.74-0.75) to 0.85 (95% CI, 0.85-0.85). Comparison against clinician recognition of sepsis yielded a minor decrease in performance, with the resulting encounter-level AUROC ranging from 0.80 (95% CI, 0.79-0.81) to 0.90 (95% CI, 0.89-0.90) across sites. Positive predictive values remained low, from 0.13 (95% CI, 0.13-0.14) to 0.26 (95% CI, 0.25-0.27), with a high number needed to evaluate and high alert burden.

In this prognostic study of a new sepsis prediction model, a multicenter prospective validation performed across 4 major US health systems found improved discrimination for the early prediction of sepsis but noted high institutional variability, low positive predictive value, and high alert burden.