hSNMF: Hybrid Spatially Regularized NMF for Image-Derived Spatial Transcriptomics

TL;DR

This paper introduces hSNMF, a novel hybrid spatially regularized NMF method that improves clustering and biological coherence in high-dimensional spatial transcriptomics data from Xenium platform images.

Contribution

The paper develops and benchmarks two spatially regularized NMF variants, including the novel hSNMF, for enhanced analysis of high-resolution spatial transcriptomics data.

Findings

hSNMF outperforms baseline methods in spatial compactness and cluster separability.

The methods achieve high biological coherence in tumor microarray analysis.

Spatial regularization significantly improves clustering quality.

Abstract

High-resolution spatial transcriptomics platforms, such as Xenium, generate single-cell images that capture both molecular and spatial context, but their extremely high dimensionality poses major challenges for representation learning and clustering. In this study, we analyze data from the Xenium platform, which captures high-resolution images of tumor microarray (TMA) tissues and converts them into cell-by-gene matrices suitable for computational analysis. We benchmark and extend nonnegative matrix factorization (NMF) for spatial transcriptomics by introducing two spatially regularized variants. First, we propose Spatial NMF (SNMF), a lightweight baseline that enforces local spatial smoothness by diffusing each cell's NMF factor vector over its spatial neighborhood. Second, we introduce Hybrid Spatial NMF (hSNMF), which performs spatially regularized NMF followed by Leiden clustering on a hybrid adjacency that integrates spatial proximity (via a contact-radius graph) and transcriptomic similarity through a tunable mixing parameter alpha. Evaluated on a cholangiocarcinoma dataset, SNMF and hSNMF achieve markedly improved spatial compactness (CHAOS < 0.004, Moran's I > 0.96), greater cluster separability (Silhouette > 0.12, DBI < 1.8), and higher biological coherence (CMC and enrichment) compared to other spatial baselines. Availability and implementation: https://github.com/ishtyaqmahmud/hSNMF