Low Rank Approximation in the Presence of Outliers

TL;DR

This paper studies low rank approximation in PCA with outliers, proposing sampling-based algorithms that provide guarantees despite the presence of outliers, and establishing the necessity of bi-criteria solutions.

Contribution

It introduces a simple algorithm with bi-criteria guarantees for PCA with outliers and proves such guarantees are unavoidable under certain complexity assumptions.

Findings

Sampling-based methods remain effective with outliers.

The proposed algorithm achieves bi-criteria guarantees.

Bi-criteria guarantees are proven to be necessary.

Abstract

We consider the problem of principal component analysis (PCA) in the presence of outliers. Given a matrix $A$ ($d \times n$) and parameters $k, m$, the goal is to remove a set of at most $m$ columns of $A$ (known as outliers), so as to minimize the rank-$k$ approximation error of the remaining matrix. While much of the work on this problem has focused on recovery of the rank-$k$ subspace under assumptions on the inliers and outliers, we focus on the approximation problem above. Our main result shows that sampling-based methods developed in the outlier-free case give non-trivial guarantees even in the presence of outliers. Using this insight, we develop a simple algorithm that has bi-criteria guarantees. Further, unlike similar formulations for clustering, we show that bi-criteria guarantees are unavoidable for the problem, under appropriate complexity assumptions.