Validation of nonlinear PCA

TL;DR

This paper introduces a novel validation method for nonlinear PCA using missing data estimation error, effectively selecting optimal model complexity where traditional methods fail.

Contribution

It proposes a new validation approach based on missing data prediction error for nonlinear PCA, overcoming limitations of standard validation techniques.

Findings

The new validation method accurately identifies the optimal model complexity.

Standard validation techniques tend to overfit nonlinear PCA models.

The approach improves model selection in unsupervised learning contexts.

Abstract

Linear principal component analysis (PCA) can be extended to a nonlinear PCA by using artificial neural networks. But the benefit of curved components requires a careful control of the model complexity. Moreover, standard techniques for model selection, including cross-validation and more generally the use of an independent test set, fail when applied to nonlinear PCA because of its inherent unsupervised characteristics. This paper presents a new approach for validating the complexity of nonlinear PCA models by using the error in missing data estimation as a criterion for model selection. It is motivated by the idea that only the model of optimal complexity is able to predict missing values with the highest accuracy. While standard test set validation usually favours over-fitted nonlinear PCA models, the proposed model validation approach correctly selects the optimal model complexity.