GiDR-DUN; Gradient Dimensionality Reduction -- Differences and Unification

TL;DR

This paper introduces GiDR-DUN, a new dimensionality reduction algorithm that unifies TSNE and UMAP, enabling faster embeddings and flexible switching between the two by adjusting a single parameter.

Contribution

The paper presents a theoretical and experimental analysis revealing that TSNE and UMAP differ mainly by a normalization parameter, leading to a unified method that is faster and adaptable.

Findings

GiDR-DUN can replicate TSNE and UMAP results by adjusting normalization.

GiDR-DUN performs optimization faster than existing UMAP and TSNE methods.

The normalization parameter controls the switch between TSNE and UMAP behaviors.

Abstract

TSNE and UMAP are two of the most popular dimensionality reduction algorithms due to their speed and interpretable low-dimensional embeddings. However, while attempts have been made to improve on TSNE's computational complexity, no existing method can obtain TSNE embeddings at the speed of UMAP. In this work, we show that this is indeed possible by combining the two approaches into a single method. We theoretically and experimentally evaluate the full space of parameters in the TSNE and UMAP algorithms and observe that a single parameter, the normalization, is responsible for switching between them. This, in turn, implies that a majority of the algorithmic differences can be toggled without affecting the embeddings. We discuss the implications this has on several theoretic claims underpinning the UMAP framework, as well as how to reconcile them with existing TSNE interpretations. Based on our analysis, we propose a new dimensionality reduction algorithm, GDR, that combines previously incompatible techniques from TSNE and UMAP and can replicate the results of either algorithm by changing the normalization. As a further advantage, GDR performs the optimization faster than available UMAP methods and thus an order of magnitude faster than available TSNE methods. Our implementation is plug-and-play with the traditional UMAP and TSNE libraries and can be found at github.com/Andrew-Draganov/GiDR-DUN.