Gradient Methods with Dynamic Inexact Oracles

TL;DR

This paper introduces the concept of dynamic inexact oracles to analyze primal-dual gradient methods for convex-concave minimax problems, enabling the development of accelerated algorithms with proven convergence.

Contribution

It models approximate gradient computations via dynamic inexact oracles and provides a unified convergence analysis for these systems, leading to new accelerated primal-dual algorithms.

Findings

Unified convergence analysis for dynamic inexact oracles

Development of new accelerated primal-dual algorithms

Demonstration of the approach's effectiveness in convex-concave minimax problems

Abstract

We show that the primal-dual gradient method, also known as the gradient descent ascent method, for solving convex-concave minimax problems can be viewed as an inexact gradient method applied to the primal problem. The gradient, whose exact computation relies on solving the inner maximization problem, is computed approximately by another gradient method. To model the approximate computational routine implemented by iterative algorithms, we introduce the notion of dynamic inexact oracles, which are discrete-time dynamical systems whose output asymptotically approaches the output of an exact oracle. We present a unified convergence analysis for dynamic inexact oracles realized by general first-order methods and demonstrate its use in creating new accelerated primal-dual algorithms.