Integrating Pathology Foundation Models and Spatial Transcriptomics for Cellular Decomposition from Histology Images

TL;DR

This paper introduces a lightweight, efficient method that leverages pathology foundation models to accurately predict cellular composition from histology images, bypassing the need for costly spatial transcriptomics.

Contribution

The work presents a novel approach combining foundation model features with a simple MLP to estimate cell types directly from images, reducing computational costs.

Findings

Achieves accurate cell-type predictions comparable to existing methods.

Reduces computational complexity significantly.

Demonstrates effectiveness on histology images without additional spatial transcriptomics data.

Abstract

The rapid development of digital pathology and modern deep learning has facilitated the emergence of pathology foundation models that are expected to solve general pathology problems under various disease conditions in one unified model, with or without fine-tuning. In parallel, spatial transcriptomics has emerged as a transformative technology that enables the profiling of gene expression on hematoxylin and eosin (H&E) stained histology images. Spatial transcriptomics unlocks the unprecedented opportunity to dive into existing histology images at a more granular, cellular level. In this work, we propose a lightweight and training-efficient approach to predict cellular composition directly from H&E-stained histology images by leveraging information-enriched feature embeddings extracted from pre-trained pathology foundation models. By training a lightweight multi-layer perceptron (MLP) regressor on cell-type abundances derived via cell2location, our method efficiently distills knowledge from pathology foundation models and demonstrates the ability to accurately predict cell-type compositions from histology images, without physically performing the costly spatial transcriptomics. Our method demonstrates competitive performance compared to existing methods such as Hist2Cell, while significantly reducing computational complexity.