Non-linear PCA via Evolution Strategies: a Novel Objective Function

TL;DR

This paper introduces a non-linear PCA method that uses neural networks and Evolution Strategies to improve variance explanation and interpretability over traditional PCA and kernel PCA, handling categorical data effectively.

Contribution

It proposes a novel non-linear PCA framework with a new objective function optimized by Evolution Strategies, combining interpretability with non-linear flexibility.

Findings

Outperforms linear PCA and kernel PCA in explained variance.

Preserves interpretability similar to PCA.

Handles categorical and ordinal variables without dimensional explosion.

Abstract

Principal Component Analysis (PCA) is a powerful and popular dimensionality reduction technique. However, due to its linear nature, it often fails to capture the complex underlying structure of real-world data. While Kernel PCA (kPCA) addresses non-linearity, it sacrifices interpretability and struggles with hyperparameter selection. In this paper, we propose a robust non-linear PCA framework that unifies the interpretability of PCA with the flexibility of neural networks. Our method parametrizes variable transformations via neural networks, optimized using Evolution Strategies (ES) to handle the non-differentiability of eigendecomposition. We introduce a novel, granular objective function that maximizes the individual variance contribution of each variable providing a stronger learning signal than global variance maximization. This approach natively handles categorical and ordinal variables without the dimensional explosion associated with one-hot encoding. We demonstrate that our method significantly outperforms both linear PCA and kPCA in explained variance across synthetic and real-world datasets. At the same time, it preserves PCA's interpretability, enabling visualization and analysis of feature contributions using standard tools such as biplots. The code can be found on GitHub.