# On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding

## TL;DR

This paper introduces a new theoretical framework linking nonlinear dimensionality reduction (NLDR) to linear smoothing, enabling better understanding, comparison, and development of NLDR algorithms through differential equations.

## Contribution

It establishes a novel connection between NLDR methods and linear smoothers, providing a unified analysis and deriving new algorithms based on differential operators.

## Key findings

- Existing NLDR methods asymptotically approximate solutions to differential equations.
- Many NLDR techniques can be characterized by three limiting differential operators.
- Local Tangent Space Alignment is shown to be superior among certain methods.

## Abstract

We develop theory for nonlinear dimensionality reduction (NLDR). A number of NLDR methods have been developed, but there is limited understanding of how these methods work and the relationships between them. There is limited basis for using existing NLDR theory for deriving new algorithms. We provide a novel framework for analysis of NLDR via a connection to the statistical theory of linear smoothers. This allows us to both understand existing methods and derive new ones. We use this connection to smoothing to show that asymptotically, existing NLDR methods correspond to discrete approximations of the solutions of sets of differential equations given a boundary condition. In particular, we can characterize many existing methods in terms of just three limiting differential operators and boundary conditions. Our theory also provides a way to assert that one method is preferable to another; indeed, we show Local Tangent Space Alignment is superior within a class of methods that assume a global coordinate chart defines an isometric embedding of the manifold.