Performance Estimation for Smooth and Strongly Convex Sets

TL;DR

This paper extends performance estimation techniques to smooth and strongly convex sets, providing a unified framework for analyzing optimization algorithms over such sets with various oracles.

Contribution

It introduces interpolation theorems for convex sets, unifies prior function interpolation results, and offers finite-dimensional formulations for performance estimation problems.

Findings

Identifies the minimax optimal separating hyperplane method.

Reveals areas for improvement in Frank-Wolfe and Alternating Projections.

Provides numerically supported conjectures and simple theorems.

Abstract

We extend recent computer-assisted design and analysis techniques for first-order optimization over structured functions--known as performance estimation--to apply to structured sets. We prove "interpolation theorems" for smooth and strongly convex sets with Slater points and bounded diameter, showing a wide range of extremal questions amount to structured mathematical programs. Prior function interpolation theorems are recovered as a limit of our set interpolation theory. Our theory provides finite-dimensional formulations of performance estimation problems for algorithms utilizing separating hyperplane oracles, linear optimization oracles, and/or projection oracles of smooth/strongly convex sets. As direct applications of this computer-assisted machinery, we identify the minimax optimal separating hyperplane method and several areas for improvement in the theory of Frank-Wolfe, Alternating Projections, and non-Lipschitz Smooth Optimization. While particular applications and methods are not our primary focus, several simple theorems and numerically supported conjectures are provided.