Greedy construction of quadratic manifolds for nonlinear dimensionality reduction and nonlinear model reduction

TL;DR

This paper introduces a greedy method for constructing quadratic manifolds that improves nonlinear dimensionality reduction and model reduction by efficiently incorporating principal components for higher accuracy.

Contribution

The work presents a novel greedy algorithm that constructs subspaces including later principal components, enhancing quadratic correction efficiency in dimensionality reduction.

Findings

Achieves orders of magnitude higher accuracy than linear methods

Scales to data with millions of dimensions

Demonstrates effectiveness through numerical experiments

Abstract

Dimensionality reduction on quadratic manifolds augments linear approximations with quadratic correction terms. Previous works rely on linear approximations given by projections onto the first few leading principal components of the training data; however, linear approximations in subspaces spanned by the leading principal components alone can miss information that are necessary for the quadratic correction terms to be efficient. In this work, we propose a greedy method that constructs subspaces from leading as well as later principal components so that the corresponding linear approximations can be corrected most efficiently with quadratic terms. Properties of the greedily constructed manifolds allow applying linear algebra reformulations so that the greedy method scales to data points with millions of dimensions. Numerical experiments demonstrate that an orders of magnitude higher accuracy is achieved with the greedily constructed quadratic manifolds compared to manifolds that are based on the leading principal components alone.