Stochastic and Private Nonconvex Outlier-Robust PCA

TL;DR

This paper introduces stochastic geodesic gradient descent methods for outlier-robust PCA that are theoretically guaranteed to recover underlying subspaces and are effective in differentially private settings, outperforming convex approaches.

Contribution

It develops novel stochastic nonconvex algorithms with convergence guarantees for outlier-robust PCA, including a differentially private version using Gaussian noise.

Findings

Algorithms successfully recover subspaces in various regimes.

Differentially private method outperforms convex approaches.

Experimental results verify theoretical guarantees.

Abstract

We develop theoretically guaranteed stochastic methods for outlier-robust PCA. Outlier-robust PCA seeks an underlying low-dimensional linear subspace from a dataset that is corrupted with outliers. We are able to show that our methods, which involve stochastic geodesic gradient descent over the Grassmannian manifold, converge and recover an underlying subspace in various regimes through the development of a novel convergence analysis. The main application of this method is an effective differentially private algorithm for outlier-robust PCA that uses a Gaussian noise mechanism within the stochastic gradient method. Our results emphasize the advantages of the nonconvex methods over another convex approach to solving this problem in the differentially private setting. Experiments on synthetic and stylized data verify these results.