Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions

TL;DR

This paper introduces PathGen, a diffusion-based AI model that synthesizes gene expression data from histopathology images, enhancing cancer diagnosis and prognosis predictions without needing actual transcriptomic tests.

Contribution

The study presents a novel diffusion-based generative model, PathGen, that synthesizes transcriptomic data from histopathology images, enabling improved multimodal cancer prediction in clinical settings.

Findings

Significant performance improvements in cancer grading and risk estimation with synthesized data.

Synthesized gene expression features are statistically comparable to real data.

PathGen achieves state-of-the-art accuracy and interpretability in multimodal predictions.

Abstract

Emerging research has highlighted that artificial intelligence-based multimodal fusion of digital pathology and transcriptomic features can improve cancer diagnosis (grading/subtyping) and prognosis (survival risk) prediction. However, such direct fusion is impractical in clinical settings, where histopathology remains the gold standard and transcriptomic tests are rarely requested in public healthcare. We experiment on two publicly available multimodal datasets, The Cancer Genomic Atlas and the Clinical Proteomic Tumor Analysis Consortium, spanning four independent cohorts: glioma-glioblastoma, renal, uterine, and breast, and observe significant performance gains in gradation and risk estimation (p-value<0.05) when incorporating synthesized transcriptomic data with WSIs. Also, predictions using synthesized features were statistically close to those obtained with real transcriptomic data (p-value>0.05), consistently across cohorts. Here we show that with our diffusion based crossmodal generative AI model, PathGen, gene expressions synthesized from digital histopathology jointly predict cancer grading and patient survival risk with high accuracy (state-of-the-art performance), certainty (through conformal coverage guarantee) and interpretability (through distributed co-attention maps). PathGen code is available for open use on GitHub at https://github.com/Samiran-Dey/PathGen.