Attraction-Repulsion Swarming: A Generalized Framework of t-SNE via Force Normalization and Tunable Interactions

TL;DR

This paper introduces ARS visualization, a new data visualization framework based on attraction-repulsion swarming dynamics, which improves upon t-SNE by offering better stability, tunability, and simplicity through force normalization and independent parameters.

Contribution

It presents a generalized swarming-based framework for data visualization that simplifies the process and enhances control over cluster tightness and spacing, differing from traditional t-SNE methods.

Findings

ARS achieves stable visualizations without complex optimization tricks.

The method allows independent tuning of cluster tightness and spacing.

Experimental results show competitive performance with t-SNE on benchmark datasets.

Abstract

We propose a new method for data visualization based on attraction-repulsion swarming (ARS) dynamics, which we call ARS visualization. ARS is a generalized framework that is based on viewing the t-distributed stochastic neighbor embedding (t-SNE) visualization technique as a swarm of interacting agents driven by attraction and repulsion. Motivated by recent developments in swarming, we modify the t-SNE dynamics to include a normalization by the \emph{total influence}, which results in better posed dynamics in which we can use a data size independent time step (of $h=1$) and a simple iteration, without the need for the array of optimization tricks employed in t-SNE. ARS also includes the ability to separately tune the attraction and repulsion kernels, which gives the user control over the tightness within clusters and the spacing between them in the visualization. In contrast with t-SNE, our proposed ARS data visualization method is not gradient descent on the Kullback-Leibler divergence, and can be viewed solely as an interacting particle system driven by attraction and repulsion forces. We provide theoretical results illustrating how the choice of interaction kernel affects the dynamics, and experimental results to validate our method and compare to t-SNE on the MNIST and Cifar-10 data sets.