The Global Geometry of Centralized and Distributed Low-rank Matrix Recovery without Regularization

TL;DR

This paper proves that the unregularized low-rank matrix recovery problem has benign global geometry, ensuring convergence of local search algorithms, and extends these results to distributed optimization.

Contribution

It establishes that the original unregularized factorized problem shares the same benign geometry as the regularized version, closing the theory-practice gap.

Findings

Unregularized problem has no spurious local minima.

Benign global geometry extends to distributed optimization.

Convergence guarantees for local search algorithms.

Abstract

Low-rank matrix recovery is a fundamental problem in signal processing and machine learning. A recent very popular approach to recovering a low-rank matrix X is to factorize it as a product of two smaller matrices, i.e., X = UV^T, and then optimize over U, V instead of X. Despite the resulting non-convexity, recent results have shown that many factorized objective functions actually have benign global geometry---with no spurious local minima and satisfying the so-called strict saddle property---ensuring convergence to a global minimum for many local-search algorithms. Such results hold whenever the original objective function is restricted strongly convex and smooth. However, most of these results actually consider a modified cost function that includes a balancing regularizer. While useful for deriving theory, this balancing regularizer does not appear to be necessary in practice. In this work, we close this theory-practice gap by proving that the unaltered factorized non-convex problem, without the balancing regularizer, also has similar benign global geometry. Moreover, we also extend our theoretical results to the field of distributed optimization.