A Universal Transfer Theorem for Convex Optimization Algorithms Using Inexact First-order Oracles

TL;DR

This paper introduces a universal transfer theorem that enables any convex optimization algorithm using exact first-order information to be adapted for inexact information, broadening applicability to various methods and problem structures.

Contribution

It provides a black-box framework to convert exact first-order algorithms into inexact ones, applicable to a wide range of convex and structured nonconvex optimization methods.

Findings

Universal transfer theorem for inexact first-order oracles

Applicable to diverse algorithms including projection-free and cutting-plane methods

Extends to structured nonconvex problems with mixed-integer variables

Abstract

Given any algorithm for convex optimization that uses exact first-order information (i.e., function values and subgradients), we show how to use such an algorithm to solve the problem with access to inexact first-order information. This is done in a ``black-box'' manner without knowledge of the internal workings of the algorithm. This complements previous work that considers the performance of specific algorithms like (accelerated) gradient descent with inexact information. In particular, our results apply to a wider range of algorithms beyond variants of gradient descent, e.g., projection-free methods, cutting-plane methods, or any other first-order methods formulated in the future. Further, they also apply to algorithms that handle structured nonconvexities like mixed-integer decision variables.