A Lyapunov analysis of Korpelevich's extragradient method with fast and flexible extensions

TL;DR

This paper introduces a Lyapunov-based analysis of Korpelevich's extragradient method, improving convergence guarantees and enabling flexible, adaptive extensions with potential superlinear rates, validated by numerical experiments.

Contribution

It provides a novel Lyapunov analysis that sharpens convergence guarantees and designs adaptive, flexible extragradient extensions with superlinear potential.

Findings

Achieves $o(1/k)$ last-iterate convergence rate.

Constructed Lyapunov function bounds standard optimality measures.

Extensions retain convergence and can attain superlinear rates.

Abstract

We develop a Lyapunov-based analysis of Korpelevich's extragradient method and show that it achieves an $o(1/k)$ last-iterate convergence rate of the constructed Lyapunov function. This Lyapunov function simultaneously upper bounds several standard measures of optimality, which allows our analysis to sharpen existing last-iterate convergence guarantees for these measures. Moreover, the same analysis enables the design of a class of flexible extensions of the extragradient method in which extragradient steps are adaptively blended with user-specified directions via a Lyapunov-guided line-search procedure. These extensions retain global convergence under practical assumptions and can attain superlinear rates when the directions are chosen appropriately. Numerical experiments confirm the simplicity and efficiency of the proposed framework.