Relations among Some Low Rank Subspace Recovery Models

TL;DR

This paper reveals deep connections among low rank subspace recovery models, positioning R-PCA as a central, theoretically solid, and computationally efficient approach that can be extended to other models.

Contribution

It uncovers closed-form relationships among models like R-PCA, R-LRR, and R-LatLRR, establishing R-PCA as the core method for low rank subspace recovery.

Findings

Solutions to different models can be derived from R-PCA in closed form.

R-PCA provides a solid theoretical foundation under certain conditions.

Proposed algorithms outperform state-of-the-art methods in speed and accuracy.

Abstract

Recovering intrinsic low dimensional subspaces from data distributed on them is a key preprocessing step to many applications. In recent years, there has been a lot of work that models subspace recovery as low rank minimization problems. We find that some representative models, such as Robust Principal Component Analysis (R-PCA), Robust Low Rank Representation (R-LRR), and Robust Latent Low Rank Representation (R-LatLRR), are actually deeply connected. More specifically, we discover that once a solution to one of the models is obtained, we can obtain the solutions to other models in closed-form formulations. Since R-PCA is the simplest, our discovery makes it the center of low rank subspace recovery models. Our work has two important implications. First, R-PCA has a solid theoretical foundation. Under certain conditions, we could find better solutions to these low rank models at overwhelming probabilities, although these models are non-convex. Second, we can obtain significantly faster algorithms for these models by solving R-PCA first. The computation cost can be further cut by applying low complexity randomized algorithms, e.g., our novel $\ell_{2,1}$ filtering algorithm, to R-PCA. Experiments verify the advantages of our algorithms over other state-of-the-art ones that are based on the alternating direction method.