SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space

TL;DR

SPADE is a novel foundation model that effectively integrates histopathology images with spatial transcriptomics data, creating a unified latent space that enhances molecular heterogeneity analysis and improves performance on various pathology tasks.

Contribution

This work introduces SPADE, a new model that combines histopathology and spatial transcriptomics using a mixture-of-experts approach, enabling more comprehensive molecular and morphological analysis.

Findings

SPADE outperforms baseline models in few-shot learning tasks.

The model creates an ST-informed latent space capturing molecular heterogeneity.

Pretrained on HEST-1k, SPADE demonstrates superior generalization across 20 tasks.

Abstract

The rapid growth of digital pathology and advances in self-supervised deep learning have enabled the development of foundational models for various pathology tasks across diverse diseases. While multimodal approaches integrating diverse data sources have emerged, a critical gap remains in the comprehensive integration of whole-slide images (WSIs) with spatial transcriptomics (ST), which is crucial for capturing critical molecular heterogeneity beyond standard hematoxylin & eosin (H&E) staining. We introduce SPADE, a foundation model that integrates histopathology with ST data to guide image representation learning within a unified framework, in effect creating an ST-informed latent space. SPADE leverages a mixture-of-data experts technique, where experts are created via two-stage imaging feature-space clustering using contrastive learning to learn representations of co-registered WSI patches and gene expression profiles. Pre-trained on the comprehensive HEST-1k dataset, SPADE is evaluated on 20 downstream tasks, demonstrating significantly superior few-shot performance compared to baseline models, highlighting the benefits of integrating morphological and molecular information into one latent space. Code and pretrained weights are available at https://github.com/uclabair/SPADE.