Uniform Manifold Approximation and Projection (UMAP) and its Variants: Tutorial and Survey

TL;DR

This paper provides a comprehensive tutorial and survey of UMAP, a leading dimensionality reduction technique, covering its algorithms, theoretical foundations, variants, and comparisons with other methods.

Contribution

It offers an in-depth explanation of UMAP's algorithm, theory, and variants, including supervised, density-preserving, parametric, and progressive versions.

Findings

UMAP outperforms t-SNE and LargeVis in visualization quality.

Various UMAP variants enable density preservation, deep learning embedding, and streaming data handling.

Theoretical foundations connect UMAP to algebraic topology and category theory.

Abstract

Uniform Manifold Approximation and Projection (UMAP) is one of the state-of-the-art methods for dimensionality reduction and data visualization. This is a tutorial and survey paper on UMAP and its variants. We start with UMAP algorithm where we explain probabilities of neighborhood in the input and embedding spaces, optimization of cost function, training algorithm, derivation of gradients, and supervised and semi-supervised embedding by UMAP. Then, we introduce the theory behind UMAP by algebraic topology and category theory. Then, we introduce UMAP as a neighbor embedding method and compare it with t-SNE and LargeVis algorithms. We discuss negative sampling and repulsive forces in UMAP's cost function. DensMAP is then explained for density-preserving embedding. We then introduce parametric UMAP for embedding by deep learning and progressive UMAP for streaming and out-of-sample data embedding.