MM-SurvNet: Deep Learning-Based Survival Risk Stratification in Breast Cancer Through Multimodal Data Fusion

TL;DR

This paper introduces MM-SurvNet, a deep learning model that combines multimodal data including images, genetics, and clinical information to improve survival risk prediction in breast cancer patients.

Contribution

It presents a novel fusion architecture using vision transformers and self-attention mechanisms to integrate diverse data types for better prognostic accuracy.

Findings

Achieved a mean C-index of 0.64 on TCGA-BRCA dataset.

Outperformed existing survival prediction methods.

Demonstrated the effectiveness of multimodal data fusion in clinical prognosis.

Abstract

Survival risk stratification is an important step in clinical decision making for breast cancer management. We propose a novel deep learning approach for this purpose by integrating histopathological imaging, genetic and clinical data. It employs vision transformers, specifically the MaxViT model, for image feature extraction, and self-attention to capture intricate image relationships at the patient level. A dual cross-attention mechanism fuses these features with genetic data, while clinical data is incorporated at the final layer to enhance predictive accuracy. Experiments on the public TCGA-BRCA dataset show that our model, trained using the negative log likelihood loss function, can achieve superior performance with a mean C-index of 0.64, surpassing existing methods. This advancement facilitates tailored treatment strategies, potentially leading to improved patient outcomes.