AMMASurv: Asymmetrical Multi-Modal Attention for Accurate Survival Analysis with Whole Slide Images and Gene Expression Data

TL;DR

AMMASurv introduces an asymmetrical multi-modal attention model that enhances survival analysis by effectively leveraging intrinsic information within each modality and flexibly adjusting to their varying importance, outperforming existing methods.

Contribution

The paper proposes a novel asymmetrical multi-modal attention approach that improves survival prediction accuracy by better utilizing intra-modality information and modality importance.

Findings

AMMASurv outperforms state-of-the-art methods in survival prediction.

The model effectively captures intrinsic modality information.

Flexible adaptation to modality importance improves results.

Abstract

The use of multi-modal data such as the combination of whole slide images (WSIs) and gene expression data for survival analysis can lead to more accurate survival predictions. Previous multi-modal survival models are not able to efficiently excavate the intrinsic information within each modality. Moreover, previous methods regard the information from different modalities as similarly important so they cannot flexibly utilize the potential connection between the modalities. To address the above problems, we propose a new asymmetrical multi-modal method, termed as AMMASurv. Different from previous works, AMMASurv can effectively utilize the intrinsic information within every modality and flexibly adapts to the modalities of different importance. Encouraging experimental results demonstrate the superiority of our method over other state-of-the-art methods.