Towards One Model for Classical Dimensionality Reduction: A Probabilistic Perspective on UMAP and t-SNE

TL;DR

This paper provides a probabilistic reinterpretation of UMAP and t-SNE as MAP inference in a model based on the graph Laplacian, linking them to Gaussian process models and offering new theoretical insights.

Contribution

It introduces a probabilistic framework that unifies UMAP and t-SNE, connecting them to Gaussian process models and enabling new analytical tools.

Findings

UMAP and t-SNE can be approximated as MAP inference in a probabilistic model.

The model relates graph Laplacians to Wishart distributions and Gaussian processes.

New tools are proposed for studying similar dimensionality reduction methods.

Abstract

This paper shows that dimensionality reduction methods such as UMAP and t-SNE, can be approximately recast as MAP inference methods corresponding to a model introduced in Ravuri et al. (2023), that describes the graph Laplacian (an estimate of the data precision matrix) using a Wishart distribution, with a mean given by a non-linear covariance function evaluated on the latents. This interpretation offers deeper theoretical and semantic insights into such algorithms, and forging a connection to Gaussian process latent variable models by showing that well-known kernels can be used to describe covariances implied by graph Laplacians. We also introduce tools with which similar dimensionality reduction methods can be studied.