Factor Group-Sparse Regularization for Efficient Low-Rank Matrix Recovery

TL;DR

This paper introduces a new nonconvex regularizer for low-rank matrix recovery that is sharper than nuclear norm and enables efficient optimization without SVD, improving accuracy and robustness.

Contribution

The paper proposes factor group-sparse regularizers as a novel nonconvex approach for low-rank matrix recovery, avoiding SVD and providing better theoretical and empirical performance.

Findings

Sharper than nuclear norm regularization

Improved generalization error bounds for matrix completion

Effective in low-rank matrix completion and robust PCA

Abstract

This paper develops a new class of nonconvex regularizers for low-rank matrix recovery. Many regularizers are motivated as convex relaxations of the matrix rank function. Our new factor group-sparse regularizers are motivated as a relaxation of the number of nonzero columns in a factorization of the matrix. These nonconvex regularizers are sharper than the nuclear norm; indeed, we show they are related to Schatten-$p$ norms with arbitrarily small $0 < p \leq 1$. Moreover, these factor group-sparse regularizers can be written in a factored form that enables efficient and effective nonconvex optimization; notably, the method does not use singular value decomposition. We provide generalization error bounds for low-rank matrix completion which show improved upper bounds for Schatten-$p$ norm reglarization as $p$ decreases. Compared to the max norm and the factored formulation of the nuclear norm, factor group-sparse regularizers are more efficient, accurate, and robust to the initial guess of rank. Experiments show promising performance of factor group-sparse regularization for low-rank matrix completion and robust principal component analysis.