Auto-associative models, nonlinear Principal component analysis, manifolds and projection pursuit

TL;DR

This paper introduces auto-associative models as a generalization of PCA, utilizing a manifold approximation approach via projection pursuit, with theoretical guarantees and practical comparisons to PCA and neural networks.

Contribution

It proposes a new class of auto-associative models for manifold approximation, with theoretical properties and efficient implementation, extending PCA and neural network methods.

Findings

The residual norm does not increase during the algorithm

The algorithm converges in a finite number of steps

Auto-associative models outperform PCA in certain cases

Abstract

In this paper, auto-associative models are proposed as candidates to the generalization of Principal Component Analysis. We show that these models are dedicated to the approximation of the dataset by a manifold. Here, the word "manifold" refers to the topology properties of the structure. The approximating manifold is built by a projection pursuit algorithm. At each step of the algorithm, the dimension of the manifold is incremented. Some theoretical properties are provided. In particular, we can show that, at each step of the algorithm, the mean residuals norm is not increased. Moreover, it is also established that the algorithm converges in a finite number of steps. Some particular auto-associative models are exhibited and compared to the classical PCA and some neural networks models. Implementation aspects are discussed. We show that, in numerous cases, no optimization procedure is required. Some illustrations on simulated and real data are presented.