Translating Machine Learning Interpretability into Clinical Insights for ICU Mortality Prediction

TL;DR

This study develops and evaluates machine learning models for ICU mortality prediction, emphasizing interpretability to translate complex model insights into actionable clinical knowledge across diverse settings.

Contribution

It introduces a framework for interpreting ML models in ICU mortality prediction, addressing variability and the Rashomon effect to improve clinical applicability.

Findings

Random forest AUROC of 0.912 and 0.839 across datasets

Consistent predictors identified across models and datasets

Alignment with routine clinical variables enhances interpretability

Abstract

Current research efforts largely focus on employing at most one interpretable method to elucidate machine learning (ML) model performance. However, significant barriers remain in translating these interpretability techniques into actionable insights for clinicians, notably due to complexities such as variability across clinical settings and the Rashomon effect. In this study, we developed and rigorously evaluated two ML models along with interpretation mechanisms, utilizing data from 131,051 ICU admissions across 208 hospitals in the United States, sourced from the eICU Collaborative Research Database. We examined two datasets: one with imputed missing values (130,810 patients, 5.58% ICU mortality) and another excluding patients with missing data (5,661 patients, 23.65% ICU mortality). The random forest (RF) model demonstrated an AUROC of 0.912 with the first dataset and 0.839 with the second dataset, while the XGBoost model achieved an AUROC of 0.924 with the first dataset and 0.834 with the second dataset. Consistently identified predictors of ICU mortality across datasets, cross-validation folds, models, and explanation mechanisms included lactate levels, arterial pH, body temperature, and others. By aligning with routinely collected clinical variables, this study aims to enhance ML model interpretability for clinical use, promote greater understanding and adoption among clinicians, and ultimately contribute to improved patient outcomes.