Deep Diffusion Maps

TL;DR

This paper introduces a deep learning-based approach to approximate Diffusion Maps embeddings, enabling scalable, out-of-sample dimensionality reduction without spectral decomposition, thus addressing key limitations of traditional manifold learning methods.

Contribution

The authors reformulate Diffusion Maps as an unconstrained minimization problem and train neural networks to compute embeddings efficiently, including for data outside the training set.

Findings

Outperforms traditional Diffusion Maps on various datasets

Reduces computational complexity and memory usage

Enables out-of-sample embedding without spectral decomposition

Abstract

One of the fundamental problems within the field of machine learning is dimensionality reduction. Dimensionality reduction methods make it possible to combat the so-called curse of dimensionality, visualize high-dimensional data and, in general, improve the efficiency of storing and processing large data sets. One of the best-known nonlinear dimensionality reduction methods is Diffusion Maps. However, despite their virtues, both Diffusion Maps and many other manifold learning methods based on the spectral decomposition of kernel matrices have drawbacks such as the inability to apply them to data outside the initial set, their computational complexity, and high memory costs for large data sets. In this work, we propose to alleviate these problems by resorting to deep learning. Specifically, a new formulation of Diffusion Maps embedding is offered as a solution to a certain unconstrained minimization problem and, based on it, a cost function to train a neural network which computes Diffusion Maps embedding -- both inside and outside the training sample -- without the need to perform any spectral decomposition. The capabilities of this approach are compared on different data sets, both real and synthetic, with those of Diffusion Maps and the Nystrom method.