Relative Lipschitzness in Extragradient Methods and a Direct Recipe for Acceleration

TL;DR

This paper demonstrates that extragradient methods can achieve optimal accelerated convergence rates for smooth convex minimization by introducing the concept of relative Lipschitzness, which generalizes to local and randomized settings.

Contribution

It provides a new characterization of extragradient method convergence via relative Lipschitzness and extends this framework to randomized and local variants, enabling new acceleration results.

Findings

Extragradient methods attain optimal accelerated rates for smooth convex functions.

The framework of relative Lipschitzness generalizes to local and randomized scenarios.

New complexity bounds for box-constrained $\ell_ extbf{infty}$ regression are established.

Abstract

We show that standard extragradient methods (i.e. mirror prox and dual extrapolation) recover optimal accelerated rates for first-order minimization of smooth convex functions. To obtain this result we provide a fine-grained characterization of the convergence rates of extragradient methods for solving monotone variational inequalities in terms of a natural condition we call relative Lipschitzness. We further generalize this framework to handle local and randomized notions of relative Lipschitzness and thereby recover rates for box-constrained $\ell_\infty$ regression based on area convexity and complexity bounds achieved by accelerated (randomized) coordinate descent for smooth convex function minimization.