MMSF: Multitask and Multimodal Supervised Framework for WSI Classification and Survival Analysis
Chengying She, Chengwei Chen, Xinran Zhang, Ben Wang, Lizhuang Liu, Chengwei Shao, Yun Bian

TL;DR
This paper introduces MMSF, a multitask and multimodal framework for whole slide image classification and survival analysis that effectively integrates heterogeneous pathological and clinical data, improving accuracy and prognostic performance.
Contribution
MMSF is a novel framework that explicitly decomposes and fuses multimodal signals using a graph-based approach and multitask learning, advancing multimodal computational pathology.
Findings
Achieved 2.1-6.6% accuracy improvements over baselines.
Improved AUC by 2.2-6.9% on classification tasks.
Enhanced C-index by 7.1-9.8% for survival analysis.
Abstract
Multimodal evidence is critical in computational pathology: gigapixel whole slide images capture tumor morphology, while patient-level clinical descriptors preserve complementary context for prognosis. Integrating such heterogeneous signals remains challenging because feature spaces exhibit distinct statistics and scales. We introduce MMSF, a multitask and multimodal supervised framework built on a linear-complexity MIL backbone that explicitly decomposes and fuses cross-modal information. MMSF comprises a graph feature extraction module embedding tissue topology at the patch level, a clinical data embedding module standardizing patient attributes, a feature fusion module aligning modality-shared and modality-specific representations, and a Mamba-based MIL encoder with multitask prediction heads. Experiments on CAMELYON16 and TCGA-NSCLC demonstrate 2.1--6.6\% accuracy and 2.2--6.9\% AUC…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAI in cancer detection · Advanced Neural Network Applications · Medical Imaging and Analysis
