Continually Evolved Multimodal Foundation Models for Cancer Prognosis

TL;DR

This paper introduces a continually evolving multimodal foundation model for cancer prognosis that adaptively integrates diverse data sources, improving prediction robustness and generalizability across different hospital datasets.

Contribution

It presents a novel framework that dynamically incorporates new data modalities and distributions, overcoming limitations of static multimodal models in cancer prognosis.

Findings

Enhanced prediction accuracy on TCGA dataset

Robustness to data distribution shifts

Effective integration of diverse modalities

Abstract

Cancer prognosis is a critical task that involves predicting patient outcomes and survival rates. To enhance prediction accuracy, previous studies have integrated diverse data modalities, such as clinical notes, medical images, and genomic data, leveraging their complementary information. However, existing approaches face two major limitations. First, they struggle to incorporate newly arrived data with varying distributions into training, such as patient records from different hospitals, thus rendering sub-optimal generalizability and limited utility in real-world applications. Second, most multimodal integration methods rely on simplistic concatenation or task-specific pipelines, which fail to capture the complex interdependencies across modalities. To address these, we propose a continually evolving multi-modal foundation model. Extensive experiments on the TCGA dataset demonstrate the effectiveness of our approach, highlighting its potential to advance cancer prognosis by enabling robust and adaptive multimodal integration.