Multi-modal Data Binding for Survival Analysis Modeling with Incomplete Data and Annotations

TL;DR

This paper presents a novel multi-modal survival analysis framework that effectively handles incomplete data and censored labels, improving prediction accuracy in cancer research by integrating advanced foundation models and uncertainty estimation.

Contribution

It introduces a new method that manages incomplete multi-modal data and censored labels simultaneously, enhancing survival prediction accuracy with universal representation and pseudo-labeling.

Findings

Achieves high prediction accuracy on survival analysis datasets.

Effectively handles incomplete data and censored labels.

Utilizes foundation models for multi-modal data fusion.

Abstract

Survival analysis stands as a pivotal process in cancer treatment research, crucial for predicting patient survival rates accurately. Recent advancements in data collection techniques have paved the way for enhancing survival predictions by integrating information from multiple modalities. However, real-world scenarios often present challenges with incomplete data, particularly when dealing with censored survival labels. Prior works have addressed missing modalities but have overlooked incomplete labels, which can introduce bias and limit model efficacy. To bridge this gap, we introduce a novel framework that simultaneously handles incomplete data across modalities and censored survival labels. Our approach employs advanced foundation models to encode individual modalities and align them into a universal representation space for seamless fusion. By generating pseudo labels and incorporating uncertainty, we significantly enhance predictive accuracy. The proposed method demonstrates outstanding prediction accuracy in two survival analysis tasks on both employed datasets. This innovative approach overcomes limitations associated with disparate modalities and improves the feasibility of comprehensive survival analysis using multiple large foundation models.