Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment

TL;DR

This paper introduces a new nonlinear dimension reduction algorithm that uses local tangent space alignment to effectively learn manifold structures from noisy, unorganized data points, with proven second-order accuracy.

Contribution

The paper presents a novel manifold learning algorithm based on local tangent space alignment and provides a detailed error analysis and practical demonstrations.

Findings

Reconstruction errors are of second-order accuracy.

Effective in 2D/3D and high-dimensional data.

Successfully applied to face images with pose and lighting variations.

Abstract

Nonlinear manifold learning from unorganized data points is a very challenging unsupervised learning and data visualization problem with a great variety of applications. In this paper we present a new algorithm for manifold learning and nonlinear dimension reduction. Based on a set of unorganized data points sampled with noise from the manifold, we represent the local geometry of the manifold using tangent spaces learned by fitting an affine subspace in a neighborhood of each data point. Those tangent spaces are aligned to give the internal global coordinates of the data points with respect to the underlying manifold by way of a partial eigendecomposition of the neighborhood connection matrix. We present a careful error analysis of our algorithm and show that the reconstruction errors are of second-order accuracy. We illustrate our algorithm using curves and surfaces both in 2D/3D and higher dimensional Euclidean spaces, and 64-by-64 pixel face images with various pose and lighting conditions. We also address several theoretical and algorithmic issues for further research and improvements.