Completed Feature Disentanglement Learning for Multimodal MRIs Analysis

TL;DR

This paper introduces a novel feature disentanglement and fusion approach for multimodal MRI analysis, recovering lost shared information and improving classification accuracy over existing methods.

Contribution

It proposes the Complete Feature Disentanglement (CFD) strategy and a Dynamic Mixture-of-Experts Fusion (DMF) module for better multimodal feature integration.

Findings

Outperforms state-of-the-art methods on three MRI datasets

Recovers shared information among multiple modalities

Enhances classification accuracy significantly

Abstract

Multimodal MRIs play a crucial role in clinical diagnosis and treatment. Feature disentanglement (FD)-based methods, aiming at learning superior feature representations for multimodal data analysis, have achieved significant success in multimodal learning (MML). Typically, existing FD-based methods separate multimodal data into modality-shared and modality-specific features, and employ concatenation or attention mechanisms to integrate these features. However, our preliminary experiments indicate that these methods could lead to a loss of shared information among subsets of modalities when the inputs contain more than two modalities, and such information is critical for prediction accuracy. Furthermore, these methods do not adequately interpret the relationships between the decoupled features at the fusion stage. To address these limitations, we propose a novel Complete Feature Disentanglement (CFD) strategy that recovers the lost information during feature decoupling. Specifically, the CFD strategy not only identifies modality-shared and modality-specific features, but also decouples shared features among subsets of multimodal inputs, termed as modality-partial-shared features. We further introduce a new Dynamic Mixture-of-Experts Fusion (DMF) module that dynamically integrates these decoupled features, by explicitly learning the local-global relationships among the features. The effectiveness of our approach is validated through classification tasks on three multimodal MRI datasets. Extensive experimental results demonstrate that our approach outperforms other state-of-the-art MML methods with obvious margins, showcasing its superior performance.