Smoothed analysis of the low-rank approach for smooth semidefinite programs

TL;DR

This paper uses smoothed analysis to show that approximate stationary points in a low-rank factorized approach to semidefinite programs are nearly optimal, especially when the problem is randomly perturbed, improving understanding of non-convex optimization in SDPs.

Contribution

It provides a theoretical guarantee that approximate solutions in a low-rank factorization are nearly optimal under smoothed analysis, extending prior work to approximate stationary points.

Findings

Approximate SOSPs are near-global optima after random perturbation.

Optimality gap can be bounded in the perturbed setting.

Results apply to SDP relaxations of phase retrieval.

Abstract

We consider semidefinite programs (SDPs) of size n with equality constraints. In order to overcome scalability issues, Burer and Monteiro proposed a factorized approach based on optimizing over a matrix Y of size $n$ by $k$ such that $X = YY^*$ is the SDP variable. The advantages of such formulation are twofold: the dimension of the optimization variable is reduced and positive semidefiniteness is naturally enforced. However, the problem in Y is non-convex. In prior work, it has been shown that, when the constraints on the factorized variable regularly define a smooth manifold, provided k is large enough, for almost all cost matrices, all second-order stationary points (SOSPs) are optimal. Importantly, in practice, one can only compute points which approximately satisfy necessary optimality conditions, leading to the question: are such points also approximately optimal? To this end, and under similar assumptions, we use smoothed analysis to show that approximate SOSPs for a randomly perturbed objective function are approximate global optima, with k scaling like the square root of the number of constraints (up to log factors). Moreover, we bound the optimality gap at the approximate solution of the perturbed problem with respect to the original problem. We particularize our results to an SDP relaxation of phase retrieval.