Interpolation Constraints for Computing Worst-Case Bounds in Performance Estimation Problems

TL;DR

This paper reviews how interpolation constraints can be used within the Performance Estimation Problem framework to compute worst-case bounds for optimization algorithms, making the problems more tractable.

Contribution

It provides a comprehensive review of recent interpolation results that enable finite-dimensional representations of infinite-dimensional objects in worst-case performance analysis.

Findings

Interpolation constraints simplify worst-case bound computations.

PEP approach is applicable beyond optimization algorithms.

Finite-dimensional representations enable tractable analysis.

Abstract

The Performance Estimation Problem (PEP) approach consists in computing worst-case performance bounds on optimization algorithms by solving an optimization problem: one maximizes an error criterion over all initial conditions allowed and all functions in a given class of interest. The maximal value is then a worst-case bound, and the maximizer provides an example reaching that worst case. This approach was introduced for optimization algorithms but could in principle be applied to many other contexts involving worst-case bounds. The key challenge is the representation of infinite-dimensional objects involved in these optimization problems such as functions, and complex or non-convex objects as linear operators and their powers, networks in decentralized optimization etc. This challenge can be resolved by interpolation constraints, which allow representing the effect of these objects on vectors of interest, rather than the whole object, leading to tractable finite dimensional problems. We review several recent interpolation results and their implications in obtaining of worst-case bounds via PEP.