How to Leverage Multimodal EHR Data for Better Medical Predictions?

TL;DR

This paper explores how integrating structured data and clinical notes from EHR improves medical prediction accuracy, demonstrating that multimodal data fusion enhances deep learning performance in healthcare tasks.

Contribution

It introduces a method to extract and integrate clinical notes with other EHR data, and studies various models for optimal data utilization in medical predictions.

Findings

Fused models outperform state-of-the-art methods without clinical notes.

Clinical notes significantly improve prediction accuracy.

Effective data fusion methods are crucial for leveraging multimodal EHR data.

Abstract

Healthcare is becoming a more and more important research topic recently. With the growing data in the healthcare domain, it offers a great opportunity for deep learning to improve the quality of medical service. However, the complexity of electronic health records (EHR) data is a challenge for the application of deep learning. Specifically, the data produced in the hospital admissions are monitored by the EHR system, which includes structured data like daily body temperature, and unstructured data like free text and laboratory measurements. Although there are some preprocessing frameworks proposed for specific EHR data, the clinical notes that contain significant clinical value are beyond the realm of their consideration. Besides, whether these different data from various views are all beneficial to the medical tasks and how to best utilize these data remain unclear. Therefore, in this paper, we first extract the accompanying clinical notes from EHR and propose a method to integrate these data, we also comprehensively study the different models and the data leverage methods for better medical task prediction. The results on two medical prediction tasks show that our fused model with different data outperforms the state-of-the-art method that without clinical notes, which illustrates the importance of our fusion method and the value of clinical note features. Our code is available at https: //github.com/emnlp-mimic/mimic.