Interpretable Multimodal Learning for Cardiovascular Hemodynamics Assessment

TL;DR

This paper introduces an interpretable multimodal learning pipeline combining CMR scans and EHR data to predict PAWP, aiding non-invasive cardiovascular assessment with validated superior performance on a large dataset.

Contribution

It proposes a novel multimodal framework with tensor-based feature extraction, graph attention for EHR feature selection, and multiple fusion strategies, enhancing interpretability and prediction accuracy.

Findings

Outperforms state-of-the-art methods in PAWP prediction

Validated on 2,641 subjects from the ASPIRE registry

Pipeline is suitable for large-scale screening

Abstract

Pulmonary Arterial Wedge Pressure (PAWP) is an essential cardiovascular hemodynamics marker to detect heart failure. In clinical practice, Right Heart Catheterization is considered a gold standard for assessing cardiac hemodynamics while non-invasive methods are often needed to screen high-risk patients from a large population. In this paper, we propose a multimodal learning pipeline to predict PAWP marker. We utilize complementary information from Cardiac Magnetic Resonance Imaging (CMR) scans (short-axis and four-chamber) and Electronic Health Records (EHRs). We extract spatio-temporal features from CMR scans using tensor-based learning. We propose a graph attention network to select important EHR features for prediction, where we model subjects as graph nodes and feature relationships as graph edges using the attention mechanism. We design four feature fusion strategies: early, intermediate, late, and hybrid fusion. With a linear classifier and linear fusion strategies, our pipeline is interpretable. We validate our pipeline on a large dataset of $2,641$ subjects from our ASPIRE registry. The comparative study against state-of-the-art methods confirms the superiority of our pipeline. The decision curve analysis further validates that our pipeline can be applied to screen a large population. The code is available at https://github.com/prasunc/hemodynamics.