114. Application of a Machine Learning Algorithm to Routine Admission Laboratory Testing for Risk Adjustment among Patients with Suspected Severe Infection across 296 US Hospitals

TL;DR

This study shows that routine lab tests and age can predict in-hospital mortality for patients with severe infections, with machine learning improving accuracy.

Contribution

The study validates a machine learning model for risk adjustment in severe infection patients across 296 US hospitals.

Findings

Low bicarbonate and hypernatremia were the strongest predictors of mortality.

The XGBoost model outperformed logistic regression with a c-statistic of 0.77.

Both models showed good calibration between predicted and observed mortality risk.

Abstract

Severity of illness at the time of presentation is an important confounding variable in infectious disease research. This study validates and further develops an existing severity-of-illness model in patients with severe infection across 296 US hospitals. A prediction model at a single hospital was previously developed to estimate risk of in-hospital mortality using 9 routine admission lab tests from the University of Maryland Medical Center (PMID:31432440). We aimed to validate this modeling approach in a new patient population, and evaluate an alternative machine-learning implementation, across a national cohort.

Odds Ratios (95%) CI from Bivariate and Multivariable Models
Laboratory cut-off values aligned with those reported by Blanco et al.Calibration of Logistic Model by Risk QuintileAverage predicted probability vs. observed event rate in each quintile

Odds Ratios (95%) CI from Bivariate and Multivariable Models

Laboratory cut-off values aligned with those reported by Blanco et al.

Calibration of Logistic Model by Risk Quintile

Average predicted probability vs. observed event rate in each quintile

This retrospective cohort study included adults with suspected severe infection who were admitted to hospitals in the Premier Healthcare Database (2019–2023). Suspected severe infection was defined as the collection of blood cultures and administration of IV antibiotics according to the Sepsis-3 clinical criteria. All eligible patients who underwent 9 routine laboratory tests on the day of admission or the day before were included (Table 1): The primary outcome was in-hospital mortality. Multivariable logistic regression and XGBoost models including age and the 9 labs were trained and evaluated using a 60% training and 40% validation split. Discrimination was assessed with c-statistics, and calibration was evaluated using Brier scores and calibration plots.Calibration of XGBoost Model by Risk QuintileAverage predicted probability vs. observed event rate in each quintileXGBoost Feature ImportancePredictors ranked by average split-gain

Calibration of XGBoost Model by Risk Quintile

Average predicted probability vs. observed event rate in each quintile

XGBoost Feature Importance

Predictors ranked by average split-gain

Of 724,534 inpatients meeting inclusion criteria, 43,108 (6%) died in-hospital. In the final multivariate model (table 1), low bicarbonate (OR 2.19 [2.19–2.26]) and hypernatremia (OR 2.19 [2.13–2.26]) remained the most significant predictors of mortality. In the final multivariable logistic regression model, the c-statistic was 0.73. The XGBoost model c-statistic was 0.77. Calibration plots for both models demonstrated good agreement between predicted and observed mortality risk.

Among patients with suspected severe infection, routine admission labs and patient age accurately predicted in-hospital mortality and machine learning improved the model. This validation study provides a straightforward, broadly applicable method for severity-of-illness risk adjustment.

Anthony Harris, MD, MPH, UpToDate Wolters Kluwer Health: Infection control section editor

Figures