DR-SNE: Density-Regularized Stochastic Neighbor Embedding

TL;DR

DR-SNE introduces a density regularization term to stochastic neighbor embedding, improving density preservation and anomaly detection while maintaining neighborhood structure.

Contribution

It presents a novel density regularization approach for t-SNE that directly aligns normalized density estimates, enhancing density preservation.

Findings

DR-SNE outperforms prior methods in density preservation.

It improves anomaly detection tasks across datasets.

Maintains competitive neighborhood fidelity.

Abstract

Dimensionality reduction methods such as t-SNE are designed to preserve local neighborhood structure but do not explicitly account for how probability mass is distributed, often leading to distortions of data density. We reformulate dimensionality reduction as the joint alignment of two components: (i) conditional structure, capturing local relationships, and (ii) relative density structure, captured via local density statistics. Based on this perspective, we introduce Density-Regularized SNE (DR-SNE), which augments the stochastic neighbor embedding objective with a density regularization term derived from normalized log-density estimates. Unlike prior approaches such as DensMAP and DenSNE, which rely on local scale consistency, DR-SNE directly aligns normalized density estimates, providing a simple and scale-invariant mechanism for preserving relative density variations. Empirically, DR-SNE improves density preservation while maintaining competitive neighborhood fidelity, and yields gains on density-sensitive tasks such as anomaly detection across multiple datasets. These results suggest that incorporating density information complements geometry-focused objectives in dimensionality reduction.