# Robust and interpretable prediction of gene markers and cell types from spatial transcriptomics data

## TL;DR

STimage is a deep learning framework that predicts gene expression and cell types from histology images, combining robustness and interpretability for digital pathology.

## Contribution

STimage introduces a probabilistic framework with uncertainty quantification and interpretability for spatial transcriptomics prediction from H&E images.

## Key findings

- STimage enhances robustness by estimating gene expression distributions and quantifying uncertainty.
- The model achieves interpretability through attribution analysis at single-cell resolution.
- STimage-predicted gene expression can stratify patient survival and predict drug response.

## Abstract

Spatial transcriptomics (ST) links tissue morphology with gene expression values, opening new avenues for digital pathology. Deep learning models are used to predict gene expression or classify cell types directly from images, offering significant clinical potential but still requiring improvements in interpretability and robustness. We present STimage as a comprehensive suite of models to predict spatial gene expression and classify cell types directly from standard H&E images. STimage enhances robustness by estimating gene expression distributions and quantifying both data-driven (aleatoric) and model-based (epistemic) uncertainty using an ensemble approach with foundation models. Interpretability is achieved through attribution analysis at single-cell resolution integrated with histopathological annotations, functional genes, and latent representations. We validated STimage across diverse datasets, demonstrating its performance across various platforms. STimage-predicted gene expression can stratify patient survival and predict drug response. By enabling molecular and cellular prediction from routine histology, STimage offers a powerful tool to advance digital pathology.

Spatial transcriptomics (ST) technologies link tissue morphology with gene expression, but remain expensive to use; furthermore, models that predict ST data from histopathology images possess considerable limitations. Here, the authors develop STimage, a deep learning probabilistic framework for ST prediction from histopathology images while prioritising robustness and interpretability.