Sharp recovery and landscape guarantees for the nonconvex matrix LASSO

TL;DR

This paper establishes sharp theoretical guarantees for the nonconvex matrix LASSO, showing when second-order critical points yield optimal low-rank matrix recovery under RIP, especially in overparametrized regimes.

Contribution

It provides the first sharp, statistically optimal theory for second-order critical points of the nonconvex matrix LASSO under RIP, including overparametrized cases.

Findings

Recovery error bounds interpolate between convex and unregularized nonconvex rates.

Rank overparametrization does not always improve the optimization landscape under RIP.

Second-order critical points can either approximate convex minima or recover exact low-rank solutions.

Abstract

Low-rank matrix recovery can be solved to statistical optimality by convex matrix optimization under the classical assumption of restricted isometry property (RIP). However, for large problems, the convex formulation is commonly replaced by a smooth rank-constrained factored nonconvex problem for which algorithmic theory typically only guarantees convergence to second-order critical points. In this paper, we develop a sharp and statistically optimal theory for second-order critical points of the factored nonconvex matrix LASSO (nuclear-norm--regularized least-squares estimator) under RIP with particular emphasis on the overparametrized regime where the search rank $r$ exceeds the ground-truth rank $r_*$. Our recovery error bounds reveal the precise role of nuclear norm regularization, interpolating between the classical convex rate and known rates for the unregularized nonconvex problem. Complementing this positive result, we give examples showing that, contrary to popular belief, rank overparametrization does not always improve the optimization landscape even under RIP. This negative result raises questions about the fundamental statistical recovery capability of rank-constrained nonconvex approaches in comparison to convex approaches which have worse computational scaling. All of our results generalize to arbitrary convex functions with nuclear-norm regularization under restricted strong convexity and smoothness. In particular, we give sharp conditions under which second-order critical points of the nonconvex problem either (1) approximately recover low-rank approximate minima of the convex problem or (2) exactly recover a low-rank global optimum if one exists.