Multimodal Fusion of Echocardiography and Electronic Health Records for the Detection of Cardiac Amyloidosis

TL;DR

This paper presents a transformer-based multimodal fusion approach combining echocardiography videos and electronic health records to improve detection of cardiac amyloidosis, achieving high accuracy with fewer data.

Contribution

The study introduces a novel multimodal fusion algorithm that effectively integrates imaging and EHR data for cardiac amyloidosis detection, reducing data requirements compared to existing methods.

Findings

Achieved AUROC of 0.94 on a small dataset of 41 patients.

Demonstrated competitive performance with less data than prior imaging-only approaches.

Validated the effectiveness of multimodal data fusion for complex cardiac diagnosis.

Abstract

Cardiac amyloidosis, a rare and highly morbid condition, presents significant challenges for detection through echocardiography. Recently, there has been a surge in proposing machine-learning algorithms to identify cardiac amyloidosis, with the majority being imaging-based deep-learning approaches that require extensive data. In this study, we introduce a novel transformer-based multimodal fusion algorithm that leverages information from both imaging and electronic health records. Specifically, our approach utilizes echocardiography videos from both the parasternal long-axis (PLAX) view and the apical 4-chamber (A4C) view along with patients' demographic data, laboratory tests, and cardiac metrics to predict the probability of cardiac amyloidosis. We evaluated our method using 5-fold cross-validation on a dataset comprising 41 patients and achieved an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.94. The experimental results demonstrate that our approach can achieve competitive results with a significantly smaller dataset compared to prior imaging-based methods that required data from thousands of patients. This underscores the potential of leveraging multimodal data to enhance diagnostic accuracy in the identification of complex cardiac conditions such as cardiac amyloidosis.