CellSymphony: Deciphering the molecular and phenotypic orchestration of cells with single-cell pathomics

TL;DR

CellSymphony is a novel multimodal framework that integrates spatial transcriptomics and histology images at single-cell resolution to improve cell type annotation and understand tissue microenvironments.

Contribution

It introduces a flexible method leveraging foundation model embeddings for multimodal data fusion in spatial transcriptomics analysis.

Findings

Achieves accurate cell type annotation.

Uncovers distinct microenvironmental niches.

Demonstrates the utility of foundation models in spatial biology.

Abstract

Xenium, a new spatial transcriptomics platform, enables subcellular-resolution profiling of complex tumor tissues. Despite the rich morphological information in histology images, extracting robust cell-level features and integrating them with spatial transcriptomics data remains a critical challenge. We introduce CellSymphony, a flexible multimodal framework that leverages foundation model-derived embeddings from both Xenium transcriptomic profiles and histology images at true single-cell resolution. By learning joint representations that fuse spatial gene expression with morphological context, CellSymphony achieves accurate cell type annotation and uncovers distinct microenvironmental niches across three cancer types. This work highlights the potential of foundation models and multimodal fusion for deciphering the physiological and phenotypic orchestration of cells within complex tissue ecosystems.