Doubly Stochastic Neighbor Embedding on Spheres

TL;DR

This paper introduces DOSNES, a spherical embedding method that normalizes similarity matrices to be doubly stochastic, effectively resolving crowding issues in data visualization by leveraging spherical geometry.

Contribution

The paper proposes a novel normalization technique for SNE that enforces doubly stochasticity and demonstrates that spherical embeddings improve visualization quality.

Findings

DOSNES outperforms existing SNE methods in visualization clarity.

Spherical embeddings mitigate crowding in high-dimensional data visualization.

Theoretical analysis supports the effectiveness of doubly stochastic normalization.

Abstract

Stochastic Neighbor Embedding (SNE) methods minimize the divergence between the similarity matrix of a high-dimensional data set and its counterpart from a low-dimensional embedding, leading to widely applied tools for data visualization. Despite their popularity, the current SNE methods experience a crowding problem when the data include highly imbalanced similarities. This implies that the data points with higher total similarity tend to get crowded around the display center. To solve this problem, we introduce a fast normalization method and normalize the similarity matrix to be doubly stochastic such that all the data points have equal total similarities. Furthermore, we show empirically and theoretically that the doubly stochasticity constraint often leads to embeddings which are approximately spherical. This suggests replacing a flat space with spheres as the embedding space. The spherical embedding eliminates the discrepancy between the center and the periphery in visualization, which efficiently resolves the crowding problem. We compared the proposed method (DOSNES) with the state-of-the-art SNE method on three real-world datasets and the results clearly indicate that our method is more favorable in terms of visualization quality.