Survey: Geometric Foundations of Data Reduction

TL;DR

This survey reviews nonlinear dimensionality reduction techniques, especially spectral manifold learning, emphasizing their geometric foundations, matrix/operator representations, and convergence behaviors in the context of data embedded on low-dimensional manifolds.

Contribution

It provides a comprehensive overview of the geometric principles, matrix/operator formulations, and convergence analysis of spectral manifold learning methods for data reduction.

Findings

Spectral manifold learning methods are grounded in geometric and topological concepts.

Matrix and operator representations clarify the structure of NLDR methods.

Convergence behaviors of these methods are discussed in a geometric framework.

Abstract

This survey is written in summer, 2016. The purpose of this survey is to briefly introduce nonlinear dimensionality reduction (NLDR) in data reduction. The first two NLDR were respectively published in Science in 2000 in which they solve the similar reduction problem of high-dimensional data endowed with the intrinsic nonlinear structure. The intrinsic nonlinear structure is always interpreted as a concept in manifolds from geometry and topology in theoretical mathematics by computer scientists and theoretical physicists. In 2001, the concept of Manifold Learning first appears as an NLDR method called Laplacian Eigenmaps. In a typical manifold learning setup, the data set, also called the observation set, is distributed on or near a low dimensional manifold M embedded in RD, which yields that each observation has a D-dimensional representation. The goal of manifold learning is to reduce these observations as a compact lower-dimensional representation based on the geometric information. The reduction procedure is called the spectral manifold learning. In this paper, we derive each spectral manifold learning with the matrix and operator representation, and we then discuss the convergence behavior of each method in a geometric uniform language. Hence, the survey is named Geometric Foundations of Data Reduction.