Structural Prognostic Event Modeling for Multimodal Cancer Survival Analysis

TL;DR

This paper introduces SlotSPE, a slot-based framework that models structural prognostic events in multimodal cancer survival analysis, improving prediction accuracy and interpretability by capturing sparse, high-level signals.

Contribution

The novel SlotSPE framework effectively encodes multimodal inputs into slots for prognostic event modeling, enhancing interaction modeling and interpretability in cancer survival prediction.

Findings

Outperforms existing methods in 8 out of 10 cancer cohorts.

Achieves an overall 2.9% improvement in survival prediction accuracy.

Provides robust predictions even with missing genomic data.

Abstract

The integration of histology images and gene profiles has shown great promise for improving survival prediction in cancer. However, current approaches often struggle to model intra- and inter-modal interactions efficiently and effectively due to the high dimensionality and complexity of the inputs. A major challenge is capturing critical prognostic events that, though few, underlie the complexity of the observed inputs and largely determine patient outcomes. These events, manifested as high-level structural signals such as spatial histologic patterns or pathway co-activations, are typically sparse, patient-specific, and unannotated, making them inherently difficult to uncover. To address this, we propose SlotSPE, a slot-based framework for structural prognostic event modeling. Specifically, inspired by the principle of factorial coding, we compress each patient's multimodal inputs into compact, modality-specific sets of mutually distinctive slots using slot attention. By leveraging these slot representations as encodings for prognostic events, our framework enables both efficient and effective modeling of complex intra- and inter-modal interactions, while also facilitating seamless incorporation of biological priors that enhance prognostic relevance. Extensive experiments on ten cancer benchmarks show that SlotSPE outperforms existing methods in 8 out of 10 cohorts, achieving an overall improvement of 2.9%. It remains robust under missing genomic data and delivers markedly improved interpretability through structured event decomposition.