EigenGame: PCA as a Nash Equilibrium

TL;DR

This paper introduces a game-theoretic perspective on PCA, modeling eigenvector computation as a Nash equilibrium problem, leading to a scalable, decentralized algorithm suitable for large datasets.

Contribution

It presents a novel PCA algorithm based on a competitive game framework, combining gradient updates with orthogonalization, enabling parallelization and scalability.

Findings

Algorithm is scalable and efficient on large image datasets.

Decentralized approach allows parallel computation.

Framework offers new insights into PCA as a differentiable game.

Abstract

We present a novel view on principal component analysis (PCA) as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function. We analyze the properties of this PCA game and the behavior of its gradient based updates. The resulting algorithm -- which combines elements from Oja's rule with a generalized Gram-Schmidt orthogonalization -- is naturally decentralized and hence parallelizable through message passing. We demonstrate the scalability of the algorithm with experiments on large image datasets and neural network activations. We discuss how this new view of PCA as a differentiable game can lead to further algorithmic developments and insights.