DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Data

TL;DR

DRIM is a novel deep learning approach that effectively learns disentangled shared and modality-specific representations from incomplete multimodal healthcare data, improving prognosis prediction and robustness to missing data.

Contribution

It introduces a new method for capturing shared and unique features in multimodal medical data, addressing data incompleteness and modality-specific information.

Findings

Outperforms state-of-the-art algorithms in glioma survival prediction

Robust to missing modalities in data

Effectively disentangles shared and modality-specific representations

Abstract

Real-life medical data is often multimodal and incomplete, fueling the growing need for advanced deep learning models capable of integrating them efficiently. The use of diverse modalities, including histopathology slides, MRI, and genetic data, offers unprecedented opportunities to improve prognosis prediction and to unveil new treatment pathways. Contrastive learning, widely used for deriving representations from paired data in multimodal tasks, assumes that different views contain the same task-relevant information and leverages only shared information. This assumption becomes restrictive when handling medical data since each modality also harbors specific knowledge relevant to downstream tasks. We introduce DRIM, a new multimodal method for capturing these shared and unique representations, despite data sparsity. More specifically, given a set of modalities, we aim to encode a representation for each one that can be divided into two components: one encapsulating patient-related information common across modalities and the other, encapsulating modality-specific details. This is achieved by increasing the shared information among different patient modalities while minimizing the overlap between shared and unique components within each modality. Our method outperforms state-of-the-art algorithms on glioma patients survival prediction tasks, while being robust to missing modalities. To promote reproducibility, the code is made publicly available at https://github.com/Lucas-rbnt/DRIM