A Cross Entropy test allows quantitative statistical comparison of t-SNE and UMAP representations

TL;DR

This paper introduces a statistical test based on cross entropy and the Kolmogorov-Smirnov test to quantitatively compare t-SNE and UMAP representations of high-dimensional biomedical data, enabling robust analysis beyond visualization.

Contribution

The authors develop a novel statistical method for comparing dimensionality-reduced datasets, addressing a gap in quantitative tools for t-SNE and UMAP outputs.

Findings

The test can distinguish biological variation from reduction artifacts.

It provides a valid distance metric for dataset comparison.

The method enables hierarchical clustering of multiple samples.

Abstract

The advent of high dimensional single cell data in the biomedical sciences has necessitated the development of dimensionality-reduction tools. t-SNE and UMAP are the two most frequently used approaches, allowing clear visualisation of highly complex single cell datasets. Despite the ubiquity of these approaches and the clear need for quantitative comparison of single cell datasets, t-SNE and UMAP have largely remained data visualisation tools due to the lack of robust statistical approaches available. Here, we have derived a statistical test for evaluating the difference between dimensionality-reduced datasets, using the Kolmogorov-Smirnov test on the distributions of cross entropy of single cells within each dataset. As the approach uses the interrelationship of single cells for comparison, the resulting statistic is robust and capable of distinguishing between true biological variation and rotational symmetry generation during dimensionality reduction. Further, the test provides a valid distance between single cell datasets, allowing the organisation of multiple samples into a dendrogram for quantitative comparison of complex datasets. These results demonstrate the largely untapped potential of dimensionality-reduction tools for biomedical data analysis beyond visualisation.