Deep Recursive Embedding for High-Dimensional Data

TL;DR

This paper introduces a deep recursive embedding (DRE) framework that combines deep neural networks with mathematical embedding rules to improve high-dimensional data visualization, scalability, and out-of-sample mapping.

Contribution

It proposes a novel deep recursive embedding method that enhances data embedding performance and scalability, outperforming existing methods like t-SNE and UMAP.

Findings

DRE improves local and global structure preservation.

DRE scales to large datasets and maps out-of-sample data.

Experiments show DRE outperforms t-SNE and UMAP.

Abstract

Embedding high-dimensional data onto a low-dimensional manifold is of both theoretical and practical value. In this paper, we propose to combine deep neural networks (DNN) with mathematics-guided embedding rules for high-dimensional data embedding. We introduce a generic deep embedding network (DEN) framework, which is able to learn a parametric mapping from high-dimensional space to low-dimensional space, guided by well-established objectives such as Kullback-Leibler (KL) divergence minimization. We further propose a recursive strategy, called deep recursive embedding (DRE), to make use of the latent data representations for boosted embedding performance. We exemplify the flexibility of DRE by different architectures and loss functions, and benchmarked our method against the two most popular embedding methods, namely, t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP). The proposed DRE method can map out-of-sample data and scale to extremely large datasets. Experiments on a range of public datasets demonstrated improved embedding performance in terms of local and global structure preservation, compared with other state-of-the-art embedding methods.