PCA by Determinant Optimization has no Spurious Local Optima

TL;DR

This paper proves that a non-convex optimization approach to PCA based on volume preservation has no spurious local optima, enabling reliable solutions for this alternative interpretation of principal components.

Contribution

The paper establishes a theoretical guarantee that volume-based PCA optimization has no spurious local optima, expanding the understanding of PCA's optimization landscape.

Findings

The non-convex volume-preserving PCA program has no spurious local optima.

Empirical solvers successfully find global optima in experiments.

Theoretical results support reliable optimization for volume-based PCA.

Abstract

Principal component analysis (PCA) is an indispensable tool in many learning tasks that finds the best linear representation for data. Classically, principal components of a dataset are interpreted as the directions that preserve most of its "energy", an interpretation that is theoretically underpinned by the celebrated Eckart-Young-Mirsky Theorem. There are yet other ways of interpreting PCA that are rarely exploited in practice, largely because it is not known how to reliably solve the corresponding non-convex optimisation programs. In this paper, we consider one such interpretation of principal components as the directions that preserve most of the "volume" of the dataset. Our main contribution is a theorem that shows that the corresponding non-convex program has no spurious local optima. We apply a number of solvers for empirical confirmation.