Spurious local minima in nonconvex sum-of-squares optimization

TL;DR

This paper investigates the existence and characterization of spurious local minima in nonconvex sum-of-squares optimization problems, providing theoretical insights, conditions for exclusion, and algorithms to avoid such points.

Contribution

It introduces a topological and algebraic framework to identify spurious minima, especially on varieties of minimal degree, and develops a path algorithm to bypass these points.

Findings

Spurious second-order stationary points exist when both dimension and codimension are greater than one for minimal degree varieties.

All stationary points on the Veronese surface related to ternary quartics are boundary points and can be simplified to binary quartics.

The proposed restricted path algorithm effectively avoids spurious minima in numerical experiments.

Abstract

We study spurious second-order stationary points and local minima in a nonconvex low-rank formulation of sum-of-squares optimization on a real variety $X$. We reformulate the problem of finding a spurious local minimum in terms of syzygies of the underlying linear series, and also bring in topological tools to study this problem. When the variety $X$ is of minimal degree, there exist spurious second-order stationary points if and only if both the dimension and the codimension of the variety are greater than one, answering a question by Legat, Yuan, and Parrilo. Moreover, for surfaces of minimal degree, we provide sufficient conditions to exclude points from being spurious local minima. In particular, all second-order stationary points associated with infinite Gram matrices on the Veronese surface, corresponding to ternary quartics, lie on the boundary and can be written as a binary quartic, up to a linear change of coordinates, complementing work by Scheiderer on decompositions of ternary quartics as a sum of three squares. For general varieties of higher degree, we give examples and characterizations of spurious second-order stationary points in the interior, together with a restricted path algorithm that avoids such points with controlled step sizes, and numerical experiment results illustrating the empirical successes on plane cubic curves and Veronese varieties.