ICFNet: Integrated Cross-modal Fusion Network for Survival Prediction

TL;DR

ICFNet is a novel multi-modal deep learning framework that integrates diverse medical data types to improve survival prediction accuracy and support personalized treatment decisions.

Contribution

The paper introduces ICFNet, a new multi-modal fusion network with specialized encoders and modules, achieving superior performance over existing methods in survival prediction.

Findings

Outperforms state-of-the-art algorithms on five TCGA datasets.

Effectively integrates histopathology, genomics, demographics, and treatment data.

Demonstrates potential for clinical decision support and personalized medicine.

Abstract

Survival prediction is a crucial task in the medical field and is essential for optimizing treatment options and resource allocation. However, current methods often rely on limited data modalities, resulting in suboptimal performance. In this paper, we propose an Integrated Cross-modal Fusion Network (ICFNet) that integrates histopathology whole slide images, genomic expression profiles, patient demographics, and treatment protocols. Specifically, three types of encoders, a residual orthogonal decomposition module and a unification fusion module are employed to merge multi-modal features to enhance prediction accuracy. Additionally, a balanced negative log-likelihood loss function is designed to ensure fair training across different patients. Extensive experiments demonstrate that our ICFNet outperforms state-of-the-art algorithms on five public TCGA datasets, including BLCA, BRCA, GBMLGG, LUAD, and UCEC, and shows its potential to support clinical decision-making and advance precision medicine. The codes are available at: https://github.com/binging512/ICFNet.