Evaluating the impact of common clinical confounders on performance of deep-learning-based sepsis risk assessment

TL;DR

This paper evaluates how clinical confounders affect a deep-learning model's ability to detect sepsis risk early in emergency departments.

Contribution

The study introduces a deep-learning model for early sepsis detection and evaluates the impact of label definitions and comorbidities on model performance.

Findings

The consensus-based model achieved 83.7% sensitivity and 80% specificity in identifying sepsis risk within 24 hours.

Infection-confirmed subgroups showed high PPV (77%), but specificity dropped in cohorts with comorbidities.

The study highlights limitations of retrospective sepsis definitions in automated detection systems.

Abstract

Early identification of sepsis in the emergency department using machine learning remains a challenging problem, primarily due to the lack of a gold standard for sepsis diagnosis, the heterogeneity in clinical presentations, and the impact of confounding conditions.

In this work, we present a deep-learning-based predictive model designed to enable early detection of patients at risk of developing sepsis, using data from the first 24 h of admission. The model is based on routine blood test results commonly performed on patients, including CBC (Complete Blood Count), CMP (Comprehensive Metabolic Panel), lipid panels, vital signs, age, and sex. To address the challenge of label uncertainty as a part of the training process, we explore two different definitions, namely, Sepsis-3 and Adult Sepsis Event. We analyze the advantages and limitations of each in the context of patient clinical parameters and comorbidities. We specifically examine how the quality of the ground truth label influences the performance of the deep learning system and evaluate the effect of a consensus-based approach that incorporates both definitions. We also evaluated the model's performance across sub-cohorts, including patients with confounding comorbidities (such as chronic kidney, liver disease, and coagulation disorders) and those with infections confirmed by billing codes.

Our results show that the consensus-based model identifies at-risk patients in the first 24 h with 83.7% sensitivity, 80% specificity, 36% PPV, 97% NPV, and an AUC of 0.9. Our cohort-wise analysis revealed a high PPV (77%) in infection-confirmed subgroups and a drop in specificity across cohorts with confounding comorbidities (47-70%).

This work highlights the limitations of retrospective sepsis definitions and underscores the need for tailored approaches in automated sepsis detection, particularly when dealing with patients with confounding comorbidities.

Figures