t-SNE, Forceful Colorings and Mean Field Limits

TL;DR

This paper introduces forceful colorings for force-based dimensionality reduction methods and analyzes the mean-field limit of t-SNE on a homogeneous cluster, revealing a novel annular embedding structure.

Contribution

It presents the concept of forceful colorings applicable to various force-based methods and derives a mean-field model predicting the t-SNE embedding shape for homogeneous clusters.

Findings

Forceful colorings provide additional features from force vectors.

t-SNE on a homogeneous cluster forms a thin annulus in the limit.

Numerical results support the mean-field predictions.

Abstract

t-SNE is one of the most commonly used force-based nonlinear dimensionality reduction methods. This paper has two contributions: the first is forceful colorings, an idea that is also applicable to other force-based methods (UMAP, ForceAtlas2,...). In every equilibrium, the attractive and repulsive forces acting on a particle cancel out: however, both the size and the direction of the attractive (or repulsive) forces acting on a particle are related to its properties: the force vector can serve as an additional feature. Secondly, we analyze the case of t-SNE acting on a single homogeneous cluster (modeled by affinities coming from the adjacency matrix of a random k-regular graph); we derive a mean-field model that leads to interesting questions in classical calculus of variations. The model predicts that, in the limit, the t-SNE embedding of a single perfectly homogeneous cluster is not a point but a thin annulus of diameter $\sim k^{-1/4} n^{-1/4}$. This is supported by numerical results. The mean field ansatz extends to other force-based dimensionality reduction methods.