The Gaussian min-max theorem in the Presence of Convexity

TL;DR

This paper extends the Gaussian min-max theorem to convex settings, enabling precise bounds on optimization problems and solutions, which improves understanding of convex algorithms in noisy signal recovery.

Contribution

It introduces convexity assumptions into the Gaussian min-max theorem, allowing for tight bounds on both optimal cost and solution norms in primary optimization problems.

Findings

Provides tight bounds on optimal cost and solution norms under convexity.

Develops a framework for analyzing the performance of convex optimization algorithms.

Enhances the theoretical understanding of signal recovery in noisy environments.

Abstract

Gaussian comparison theorems are useful tools in probability theory; they are essential ingredients in the classical proofs of many results in empirical processes and extreme value theory. More recently, they have been used extensively in the analysis of non-smooth optimization problems that arise in the recovery of structured signals from noisy linear observations. We refer to such problems as Primary Optimization (PO) problems. A prominent role in the study of the (PO) problems is played by Gordon's Gaussian min-max theorem (GMT) which provides probabilistic lower bounds on the optimal cost via a simpler Auxiliary Optimization (AO) problem. Motivated by resent work of M. Stojnic, we show that under appropriate convexity assumptions the (AO) problem allows one to tightly bound both the optimal cost, as well as the norm of the solution of the (PO). As an application, we use our result to develop a general framework to tightly characterize the performance (e.g. squared-error) of a wide class of convex optimization algorithms used in the context of noisy signal recovery.