Diffusion Maps is not Dimensionality Reduction

TL;DR

Diffusion maps are spectral tools that reveal intrinsic geometry but do not directly produce low-dimensional charts, unlike Isomap and UMAP, which explicitly recover the underlying structure.

Contribution

This paper clarifies the distinction between diffusion maps and true dimensionality reduction methods through theoretical analysis and empirical comparison.

Findings

Isomap most efficiently recovers the low-dimensional chart

UMAP provides an intermediate tradeoff

DMAP requires combining multiple diffusion modes for accuracy

Abstract

Diffusion maps (DMAP) are often used as a dimensionality-reduction tool, but more precisely they provide a spectral representation of the intrinsic geometry rather than a complete charting method. To illustrate this distinction, we study a Swiss roll with known isometric coordinates and compare DMAP, Isomap, and UMAP across latent dimensions. For each representation, we fit an oracle affine readout to the ground-truth chart and measure reconstruction error. Isomap most efficiently recovers the low-dimensional chart, UMAP provides an intermediate tradeoff, and DMAP becomes accurate only after combining multiple diffusion modes. Thus the correct chart lies in the span of diffusion coordinates, but standard DMAP do not by themselves identify the appropriate combination.