A Methodological Framework for Measuring Spatial Labeling Similarity

TL;DR

This paper introduces a comprehensive framework and a specific metric, SLAM, for measuring the similarity of spatial labelings by considering label agreement, topology, and heterogeneity, with applications in spatial transcriptomics.

Contribution

It proposes a novel methodological framework transforming spatial labelings into graphs and extracting attribute distributions, leading to the development of the SLAM metric for accurate similarity measurement.

Findings

SLAM accurately reflects labeling quality in spatial transcriptomics.

The framework considers label agreement, topology, and heterogeneity.

Experimental results show SLAM outperforms existing metrics.

Abstract

Spatial labeling assigns labels to specific spatial locations to characterize their spatial properties and relationships, with broad applications in scientific research and practice. Measuring the similarity between two spatial labelings is essential for understanding their differences and the contributing factors, such as changes in location properties or labeling methods. An adequate and unbiased measurement of spatial labeling similarity should consider the number of matched labels (label agreement), the topology of spatial label distribution, and the heterogeneous impacts of mismatched labels. However, existing methods often fail to account for all these aspects. To address this gap, we propose a methodological framework to guide the development of methods that meet these requirements. Given two spatial labelings, the framework transforms them into graphs based on location organization, labels, and attributes (e.g., location significance). The distributions of their graph attributes are then extracted, enabling an efficient computation of distributional discrepancy to reflect the dissimilarity level between the two labelings. We further provide a concrete implementation of this framework, termed Spatial Labeling Analogy Metric (SLAM), along with an analysis of its theoretical foundation, for evaluating spatial labeling results in spatial transcriptomics (ST) \textit{as per} their similarity with ground truth labeling. Through a series of carefully designed experimental cases involving both simulated and real ST data, we demonstrate that SLAM provides a comprehensive and accurate reflection of labeling quality compared to other well-established evaluation metrics. Our code is available at https://github.com/YihDu/SLAM.