DREAMS: Preserving both Local and Global Structure in Dimensionality Reduction

TL;DR

DREAMS is a novel dimensionality reduction method that effectively balances local and global data structure preservation by combining features of t-SNE and PCA, demonstrated through extensive benchmarking.

Contribution

The paper introduces DREAMS, a new method that integrates local and global structure preservation in dimensionality reduction using a regularization approach.

Findings

DREAMS outperforms existing methods in preserving data structure.

It generates embeddings that balance local and global features.

Benchmark results show superior performance across multiple datasets.

Abstract

Dimensionality reduction techniques are widely used for visualizing high-dimensional data in two dimensions. Existing methods are typically designed to preserve either local (e.g., $t$-SNE, UMAP) or global (e.g., MDS, PCA) structure of the data, but none of the established methods can represent both aspects well. In this paper, we present DREAMS (Dimensionality Reduction Enhanced Across Multiple Scales), a method that combines the local structure preservation of $t$-SNE with the global structure preservation of PCA via a simple regularization term. Our approach generates a spectrum of embeddings between the locally well-structured $t$-SNE embedding and the globally well-structured PCA embedding, efficiently balancing both local and global structure preservation. We benchmark DREAMS across eleven real-world datasets, showcasing qualitatively and quantitatively its superior ability to preserve structure across multiple scales compared to previous approaches.