Convergence analysis of t-SNE as a gradient flow for point cloud on a manifold

TL;DR

This paper provides a theoretical analysis of t-SNE, showing that the algorithm's points remain bounded during optimization and establishing the existence of a minimizer for the KL divergence objective.

Contribution

It offers a rigorous foundation for t-SNE's convergence properties by analyzing it as a gradient flow on point clouds on a manifold.

Findings

t-SNE points remain bounded during optimization

Existence of a KL divergence minimizer is proven

Analysis applies under weak convergence assumptions

Abstract

We present a theoretical foundation regarding the boundedness of the t-SNE algorithm. t-SNE employs gradient descent iteration with Kullback-Leibler (KL) divergence as the objective function, aiming to identify a set of points that closely resemble the original data points in a high-dimensional space, minimizing KL divergence. Investigating t-SNE properties such as perplexity and affinity under a weak convergence assumption on the sampled dataset, we examine the behavior of points generated by t-SNE under continuous gradient flow. Demonstrating that points generated by t-SNE remain bounded, we leverage this insight to establish the existence of a minimizer for KL divergence.