A Generalization of Principal Component Analysis

TL;DR

This paper introduces a generalized PCA framework that maximizes arbitrary convex functions of principal components, providing new algorithms and neural network solutions, with evaluations on various datasets.

Contribution

It extends PCA to arbitrary convex functions and develops gradient ascent and neural network algorithms for solution computation.

Findings

Algorithms effectively optimize generalized PCA objectives.

Kernel version solutions as fixed points of recurrent neural networks.

Successful evaluation on multiple datasets.

Abstract

Conventional principal component analysis (PCA) finds a principal vector that maximizes the sum of second powers of principal components. We consider a generalized PCA that aims at maximizing the sum of an arbitrary convex function of principal components. We present a gradient ascent algorithm to solve the problem. For the kernel version of generalized PCA, we show that the solutions can be obtained as fixed points of a simple single-layer recurrent neural network. We also evaluate our algorithms on different datasets.