From Symmetry to Geometry: Tractable Nonconvex Problems

TL;DR

This paper reviews a class of nonconvex optimization problems characterized by symmetry, where geometric structure allows for efficient solutions despite nonconvexity, with applications across imaging, signal processing, and data analysis.

Contribution

It introduces a geometric perspective on nonconvex problems with symmetry, explaining how their structure enables tractable solutions and highlighting key roles of rotational and discrete symmetries.

Findings

Symmetric local minima are copies of a ground truth solution.

Critical points include balanced superpositions with negative curvature.

Symmetry structure facilitates global optimization methods.

Abstract

As science and engineering have become increasingly data-driven, the role of optimization has expanded to touch almost every stage of the data analysis pipeline, from signal and data acquisition to modeling and prediction. The optimization problems encountered in practice are often nonconvex. While challenges vary from problem to problem, one common source of nonconvexity is nonlinearity in the data or measurement model. Nonlinear models often exhibit symmetries, creating complicated, nonconvex objective landscapes, with multiple equivalent solutions. Nevertheless, simple methods (e.g., gradient descent) often perform surprisingly well in practice. The goal of this survey is to highlight a class of tractable nonconvex problems, which can be understood through the lens of symmetries. These problems exhibit a characteristic geometric structure: local minimizers are symmetric copies of a single "ground truth" solution, while other critical points occur at balanced superpositions of symmetric copies of the ground truth, and exhibit negative curvature in directions that break the symmetry. This structure enables efficient methods to obtain global minimizers. We discuss examples of this phenomenon arising from a wide range of problems in imaging, signal processing, and data analysis. We highlight the key role of symmetry in shaping the objective landscape and discuss the different roles of rotational and discrete symmetries. This area is rich with observed phenomena and open problems; we close by highlighting directions for future research.