Towards Spectral Convergence of Locally Linear Embedding on Manifolds with Boundary

TL;DR

This paper analyzes the spectral properties of the differential operator underlying Locally Linear Embedding on manifolds with boundary, providing theoretical insights and a variational framework for eigenvalue estimation.

Contribution

It introduces a regularity condition for eigenfunctions, derives the asymptotic eigenvalues analytically, and proposes a variational approach for other manifolds.

Findings

Eigenvalues are characterized by a second-order mixed-type differential operator.

A regularity condition leads to a consistent boundary condition for eigenfunctions.

Theoretical eigenvalues match numerical predictions.

Abstract

We study the eigenvalues and eigenfunctions of a differential operator that governs the asymptotic behavior of the unsupervised learning algorithm known as Locally Linear Embedding when a large data set is sampled from an interval or disc. In particular, the differential operator is of second order, mixed-type, and degenerates near the boundary. We show that a natural regularity condition on the eigenfunctions imposes a consistent boundary condition and use the Frobenius method to estimate pointwise behavior. We then determine the limiting sequence of eigenvalues analytically and compare them to numerical predictions. Finally, we propose a variational framework for determining eigenvalues on other compact manifolds.