An empirical study of using radiology reports and images to improve ICU mortality prediction

TL;DR

This study develops a deep learning model integrating clinical data, radiology reports, and chest X-ray images to improve ICU mortality prediction, outperforming traditional scoring systems.

Contribution

It introduces a multi-modality deep learning approach combining EHR data, radiology reports, and images for enhanced ICU mortality prediction.

Findings

Model achieves C-index of 0.7829, outperforming baseline.

Radiology image features contribute 2.82% to performance.

Text features and pre-defined labels also improve accuracy.

Abstract

Background: The predictive Intensive Care Unit (ICU) scoring system plays an important role in ICU management because it predicts important outcomes, especially mortality. Many scoring systems have been developed and used in the ICU. These scoring systems are primarily based on the structured clinical data in the electronic health record (EHR), which may suffer the loss of important clinical information in the narratives and images. Methods: In this work, we build a deep learning based survival prediction model with multi-modality data to predict ICU mortality. Four sets of features are investigated: (1) physiological measurements of Simplified Acute Physiology Score (SAPS) II, (2) common thorax diseases pre-defined by radiologists, (3) BERT-based text representations, and (4) chest X-ray image features. We use the Medical Information Mart for Intensive Care IV (MIMIC-IV) dataset to evaluate the proposed model. Results: Our model achieves the average C-index of 0.7829 (95% confidence interval, 0.7620-0.8038), which substantially exceeds that of the baseline with SAPS-II features (0.7470 (0.7263-0.7676)). Ablation studies further demonstrate the contributions of pre-defined labels (2.00%), text features (2.44%), and image features (2.82%).