DeLLiriuM: A large language model for delirium prediction in the ICU using structured EHR

TL;DR

DeLLiriuM is a novel large language model designed to predict ICU delirium using structured EHR data, demonstrating superior performance across multiple large-scale hospital datasets and potentially aiding timely clinical interventions.

Contribution

This study introduces DeLLiriuM, the first LLM-based ICU delirium prediction model utilizing structured EHR data, with validation across diverse large-scale datasets showing improved accuracy.

Findings

DeLLiriuM achieved AUROC scores of 0.77 and 0.84 in external validation sets.

Outperformed existing deep learning models on multiple datasets.

Validated across 195 hospitals with over 104,000 patients.

Abstract

Delirium is an acute confusional state that has been shown to affect up to 31% of patients in the intensive care unit (ICU). Early detection of this condition could lead to more timely interventions and improved health outcomes. While artificial intelligence (AI) models have shown great potential for ICU delirium prediction using structured electronic health records (EHR), most of them have not explored the use of state-of-the-art AI models, have been limited to single hospitals, or have been developed and validated on small cohorts. The use of large language models (LLM), models with hundreds of millions to billions of parameters, with structured EHR data could potentially lead to improved predictive performance. In this study, we propose DeLLiriuM, a novel LLM-based delirium prediction model using EHR data available in the first 24 hours of ICU admission to predict the probability of a patient developing delirium during the rest of their ICU admission. We develop and validate DeLLiriuM on ICU admissions from 104,303 patients pertaining to 195 hospitals across three large databases: the eICU Collaborative Research Database, the Medical Information Mart for Intensive Care (MIMIC)-IV, and the University of Florida Health's Integrated Data Repository. The performance measured by the area under the receiver operating characteristic curve (AUROC) showed that DeLLiriuM outperformed all baselines in two external validation sets, with 0.77 (95% confidence interval 0.76-0.78) and 0.84 (95% confidence interval 0.83-0.85) across 77,543 patients spanning 194 hospitals. To the best of our knowledge, DeLLiriuM is the first LLM-based delirium prediction tool for the ICU based on structured EHR data, outperforming deep learning baselines which employ structured features and can provide helpful information to clinicians for timely interventions.