Think globally, fit locally under the Manifold Setup: Asymptotic Analysis of Locally Linear Embedding

TL;DR

This paper provides an asymptotic analysis of Locally Linear Embedding (LLE) on manifolds, revealing its limitations in approximating the Laplace-Beltrami operator and its non-Markovian nature, with insights into regularization and comparisons to other algorithms.

Contribution

The paper offers the first asymptotic analysis of LLE on general manifolds, highlighting its dependence on sampling and regularization, and clarifies its relationship with diffusion maps and local regression.

Findings

LLE may not approximate the Laplace-Beltrami operator asymptotically.

The kernel function of LLE indicates it is not a Markov process.

Regularization choice critically affects LLE's asymptotic behavior.

Abstract

Since its introduction in 2000, the locally linear embedding (LLE) has been widely applied in data science. We provide an asymptotical analysis of the LLE under the manifold setup. We show that for the general manifold, asymptotically we may not obtain the Laplace-Beltrami operator, and the result may depend on the non-uniform sampling, unless a correct regularization is chosen. We also derive the corresponding kernel function, which indicates that the LLE is not a Markov process. A comparison with the other commonly applied nonlinear algorithms, particularly the diffusion map, is provided, and its relationship with the locally linear regression is also discussed.