RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency

TL;DR

RankByGene introduces a novel gene-guided histopathology representation learning framework that uses cross-modal ranking and self-supervised distillation to improve alignment and predictive accuracy across spatial transcriptomics and histology images.

Contribution

It proposes a ranking-based alignment loss combined with self-supervised knowledge distillation to enhance cross-modal alignment in spatial transcriptomics and histology images.

Findings

Improved alignment accuracy over existing methods.

Enhanced gene expression prediction performance.

Better slide-level classification and survival analysis results.

Abstract

Spatial transcriptomics (ST) provides essential spatial context by mapping gene expression within tissue, enabling detailed study of cellular heterogeneity and tissue organization. However, aligning ST data with histology images poses challenges due to inherent spatial distortions and modality-specific variations. Existing methods largely rely on direct alignment, which often fails to capture complex cross-modal relationships. To address these limitations, we propose a novel framework that aligns gene and image features using a ranking-based alignment loss, preserving relative similarity across modalities and enabling robust multi-scale alignment. To further enhance the alignment's stability, we employ self-supervised knowledge distillation with a teacher-student network architecture, effectively mitigating disruptions from high dimensionality, sparsity, and noise in gene expression data. Extensive experiments on seven public datasets that encompass gene expression prediction, slide-level classification, and survival analysis demonstrate the efficacy of our method, showing improved alignment and predictive performance over existing methods.