M2OST: Many-to-one Regression for Predicting Spatial Transcriptomics from Digital Pathology Images

TL;DR

M2OST introduces a multi-scale, many-to-one Transformer model that leverages hierarchical pathology image data to accurately predict spatial transcriptomics, outperforming existing methods with fewer resources.

Contribution

The paper presents M2OST, a novel many-to-one regression Transformer that effectively utilizes multi-scale image features and inter-spot information for spatial transcriptomics prediction.

Findings

Achieves state-of-the-art performance on three public datasets.

Uses fewer parameters and FLOPs than existing methods.

Effectively incorporates hierarchical and inter-spot features.

Abstract

The advancement of Spatial Transcriptomics (ST) has facilitated the spatially-aware profiling of gene expressions based on histopathology images. Although ST data offers valuable insights into the micro-environment of tumors, its acquisition cost remains expensive. Therefore, directly predicting the ST expressions from digital pathology images is desired. Current methods usually adopt existing regression backbones along with patch-sampling for this task, which ignores the inherent multi-scale information embedded in the pyramidal data structure of digital pathology images, and wastes the inter-spot visual information crucial for accurate gene expression prediction. To address these limitations, we propose M2OST, a many-to-one regression Transformer that can accommodate the hierarchical structure of the pathology images via a decoupled multi-scale feature extractor. Unlike traditional models that are trained with one-to-one image-label pairs, M2OST uses multiple images from different levels of the digital pathology image to jointly predict the gene expressions in their common corresponding spot. Built upon our many-to-one scheme, M2OST can be easily scaled to fit different numbers of inputs, and its network structure inherently incorporates nearby inter-spot features, enhancing regression performance. We have tested M2OST on three public ST datasets and the experimental results show that M2OST can achieve state-of-the-art performance with fewer parameters and floating-point operations (FLOPs).