Accelerating nuclear-norm regularized low-rank matrix optimization through Burer-Monteiro decomposition

TL;DR

This paper introduces BM-Global, a fast algorithm for nuclear-norm-regularized low-rank matrix optimization that adaptively finds the optimal rank, escapes saddle points, and outperforms existing methods in speed and accuracy.

Contribution

The paper presents BM-Global, a novel algorithm that efficiently solves nuclear-norm-regularized problems by adaptively adjusting rank and escaping local minima, with provable convergence guarantees.

Findings

BM-Global effectively escapes spurious local minima.

It converges faster than state-of-the-art algorithms.

It automatically identifies the optimal rank during optimization.

Abstract

This work proposes a rapid algorithm, BM-Global, for nuclear-norm-regularized convex and low-rank matrix optimization problems. BM-Global efficiently decreases the objective value via low-cost steps leveraging the nonconvex but smooth Burer-Monteiro (BM) decomposition, while effectively escapes saddle points and spurious local minima ubiquitous in the BM form to obtain guarantees of fast convergence rates to the global optima of the original nuclear-norm-regularized problem through aperiodic inexact proximal gradient steps on it. The proposed approach adaptively adjusts the rank for the BM decomposition and can provably identify an optimal rank for the BM decomposition problem automatically in the course of optimization through tools of manifold identification. BM-Global hence also spends significantly less time on parameter tuning than existing matrix-factorization methods, which require an exhaustive search for finding this optimal rank. Extensive experiments on real-world large-scale problems of recommendation systems, regularized kernel estimation, and molecular conformation confirm that BM-Global can indeed effectively escapes spurious local minima at which existing BM approaches are stuck, and is a magnitude faster than state-of-the-art algorithms for low-rank matrix optimization problems involving a nuclear-norm regularizer. Based on this research, we have released an open-source package of the proposed BM-Global at https://www.github.com/leepei/BM-Global/.