Connecting Dots -- from Local Covariance to Empirical Intrinsic Geometry and Locally Linear Embedding

TL;DR

This paper analyzes manifold learning algorithms, EIG and LLE, highlighting how local covariance structures influence geodesic distance estimation and the importance of truncation schemes, with theoretical insights into their performance.

Contribution

It provides a theoretical analysis of EIG and LLE, emphasizing the role of local covariance and curvature, and introduces improvements in geodesic distance estimation.

Findings

EIG's geodesic estimation can be corrupted without accurate dimension estimation.

Using local covariance improves local geodesic distance accuracy.

LLE's performance depends on the truncation scheme related to curvature.

Abstract

Local covariance structure under the manifold setup has been widely applied in the machine learning society. Based on the established theoretical results, we provide an extensive study of two relevant manifold learning algorithms, empirical intrinsic geometry (EIG) and the locally linear embedding (LLE) under the manifold setup. Particularly, we show that without an accurate dimension estimation, the geodesic distance estimation by EIG might be corrupted. Furthermore, we show that by taking the local covariance matrix into account, we can more accurately estimate the local geodesic distance. When understanding LLE based on the local covariance structure, its intimate relationship with the curvature suggests a variation of LLE depending on the "truncation scheme". We provide a theoretical analysis of the variation.