Non-Convex Rank Minimization via an Empirical Bayesian Approach

TL;DR

This paper introduces an empirical Bayesian approach for non-convex rank minimization that outperforms traditional convex relaxations like nuclear norm minimization, especially in robust principal component analysis.

Contribution

It proposes a variational Bayesian method that preserves the global minima of the rank function and effectively handles local minima, improving low-rank matrix estimation.

Findings

Method outperforms nuclear norm-based approaches in experiments.

Successfully estimates low-rank matrices with sparse corruptions.

Applicable to a wide range of low-rank applications.

Abstract

In many applications that require matrix solutions of minimal rank, the underlying cost function is non-convex leading to an intractable, NP-hard optimization problem. Consequently, the convex nuclear norm is frequently used as a surrogate penalty term for matrix rank. The problem is that in many practical scenarios there is no longer any guarantee that we can correctly estimate generative low-rank matrices of interest, theoretical special cases notwithstanding. Consequently, this paper proposes an alternative empirical Bayesian procedure build upon a variational approximation that, unlike the nuclear norm, retains the same globally minimizing point estimate as the rank function under many useful constraints. However, locally minimizing solutions are largely smoothed away via marginalization, allowing the algorithm to succeed when standard convex relaxations completely fail. While the proposed methodology is generally applicable to a wide range of low-rank applications, we focus our attention on the robust principal component analysis problem (RPCA), which involves estimating an unknown low-rank matrix with unknown sparse corruptions. Theoretical and empirical evidence are presented to show that our method is potentially superior to related MAP-based approaches, for which the convex principle component pursuit (PCP) algorithm (Candes et al., 2011) can be viewed as a special case.