Parameterized Principal Component Analysis

TL;DR

This paper introduces parameterized PCA (PPCA), a novel method that models data with smoothly varying subspaces based on an extra contextual parameter, improving data approximation over traditional PCA and related methods.

Contribution

PPCA is a new manifold approximation technique that incorporates a continuous parameter to model smoothly changing subspaces, unlike ad-hoc atlases or group-based PCA methods.

Findings

PPCA outperforms PCA, sparse PCA, and IPCA in reconstructing data with known smooth functions.

PPCA achieves lower reconstruction error on artificial and real datasets.

PPCA effectively models nonlinear manifolds where traditional PCA struggles.

Abstract

When modeling multivariate data, one might have an extra parameter of contextual information that could be used to treat some observations as more similar to others. For example, images of faces can vary by age, and one would expect the face of a 40 year old to be more similar to the face of a 30 year old than to a baby face. We introduce a novel manifold approximation method, parameterized principal component analysis (PPCA) that models data with linear subspaces that change continuously according to the extra parameter of contextual information (e.g. age), instead of ad-hoc atlases. Special care has been taken in the loss function and the optimization method to encourage smoothly changing subspaces across the parameter values. The approach ensures that each observation's projection will share information with observations that have similar parameter values, but not with observations that have large parameter differences. We tested PPCA on artificial data based on known, smooth functions of an added parameter, as well as on three real datasets with different types of parameters. We compared PPCA to PCA, sparse PCA and to independent principal component analysis (IPCA), which groups observations by their parameter values and projects each group using PCA with no sharing of information for different groups. PPCA recovers the known functions with less error and projects the datasets' test set observations with consistently less reconstruction error than IPCA does. In some cases where the manifold is truly nonlinear, PCA outperforms all the other manifold approximation methods compared.