Multimodal Alignment Improves Generalizability of Genomic Biomarker Prediction in Computational Pathology

TL;DR

This paper introduces MARBLE, a multimodal contrastive pretraining method that aligns histopathology images with genomic biomarker representations, enhancing the generalizability of biomarker prediction models in computational pathology.

Contribution

The study presents a novel biologically informed multimodal pretraining strategy that improves data efficiency and out-of-distribution generalization in genomic biomarker prediction.

Findings

MARBLE improves biomarker prediction accuracy on out-of-distribution data.

The approach leverages large language and protein models for representation alignment.

Experiments on the MSK-IMPACT cohort validate the method's effectiveness.

Abstract

Computational pathology models that use digitized histopathology whole-slide images have the potential to become a cost-effective and scalable alternative to molecular assays for the prediction of genomic biomarkers, a key task in precision oncology. However, as new genomic biomarkers are discovered or quantified, large, labeled datasets must be prospectively collected to train new models. To address this challenge, we developed MARBLE, a multimodal contrastive pretraining strategy that integrates structured biomarker knowledge into representation learning of histopathology images. MARBLE aligns histopathology-derived representations with representations of genomic biomarkers generated by a large language model (LLM) and a protein language model (PLM). This biologically informed alignment enables data-efficient generalization to novel, out-of-distribution biomarkers. Using the MSK-IMPACT cohort of over 40,000 patients across multiple biomarker panel versions, we design experiments grounded in real-world data to demonstrate the value of our proposed approach.