Enhancing cause of death prediction: development and validation of machine learning models using multimodal data across multiple health-care sites

TL;DR

Researchers developed machine learning models to predict causes of death using health records and found that combining structured and unstructured data improves accuracy within institutions but struggles across different healthcare systems.

Contribution

The study introduces a novel approach combining structured and unstructured EHR data for cause of death prediction and highlights generalizability challenges across institutions.

Findings

XGBoost models using structured EHR data achieved AUCs of 0.86 and 0.80 at VUMC and MGB respectively.

Adding unstructured clinical notes improved AUCs to 0.90 and 0.92 at VUMC and MGB.

Cross-institutional validation showed significant performance degradation, indicating limited generalizability.

Abstract

To develop and validate machine learning (ML) models that predict probable cause of death (CoD) using structured electronic health record (EHR) data, unstructured clinical notes, and publicly available sources.

This multi-institutional retrospective study was conducted across Vanderbilt University Medical Center (VUMC) and Massachusetts General Brigham (MGB), including deceased patients with encounters between October 1, 2015, and January 1, 2021, and confirmed death records. The cohort included 13 708 patients from VUMC and 34 839 from MGB.The primary outcome was underlying CoD categorized into the top 15 National Center for Health Statistics rankable causes, with others grouped as “Other.” Performance was assessed using weighted area under the receiver operating characteristic curve (AUC) and F-measure.

The XGBoost model using structured EHR data alone achieved weighted AUCs of 0.86 (95% CI, 0.84-0.88) at VUMC and 0.80 (95% CI, 0.79-0.80) at MGB. Adding unstructured notes improved performance, with weighted AUCs of 0.90 (95% CI, 0.88-0.93) at VUMC and 0.92 (95% CI, 0.91-0.92) at MGB. Adding publicly available data did not further improve performance. Cross-institutional validation revealed significant performance degradation.

Models integrating structured and unstructured EHR data show strong within-institution performance but limited generalizability across healthcare systems, highlighting challenges related to institutional data heterogeneity.

Machine learning models combining structured and unstructured EHR data accurately predict CoD within institutions but perform poorly across sites. Health-care institutions may benefit from adopting robust processes for locally tailored models, and future research should focus on enhancing model generalizability while addressing unique institutional data environments.

Figures