Parameter-Free Non-Ergodic Extragradient Algorithms for Solving Monotone Variational Inequalities

TL;DR

This paper introduces parameter-free extragradient algorithms with non-asymptotic last-iterate guarantees for monotone variational inequalities, achieving improved practical performance without requiring problem-specific stepsize tuning.

Contribution

The authors develop the first parameter-free extragradient methods with last-iterate guarantees for both globally and locally Lipschitz monotone VIs, extending applicability and robustness.

Findings

Achieve $o(1/\sqrt{T})$ last-iterate convergence rate for globally Lipschitz operators.

Extend the framework to locally Lipschitz operators using backtracking line search.

Demonstrate strong empirical performance on various problems, outperforming existing methods.

Abstract

Monotone variational inequalities (VIs) provide a unifying framework for convex minimization, equilibrium computation, and convex-concave saddle-point problems. Extragradient-type methods are among the most effective first-order algorithms for such problems, but their performance hinges critically on stepsize selection. While most existing theory focuses on ergodic averages of the iterates, practical performance is often driven by the significantly stronger behavior of the last iterate. Moreover, available last-iterate guarantees typically rely on fixed stepsizes chosen using problem-specific global smoothness information, which is often difficult to estimate accurately and may not even be applicable. In this paper, we develop parameter-free extragradient methods with non-asymptotic last-iterate guarantees for constrained monotone VIs. For globally Lipschitz operators, our algorithm achieves an $o(1/\sqrt{T})$ last-iterate rate. We then extend the framework to locally Lipschitz operators via backtracking line search and obtain the same rate while preserving parameter-freeness, thereby making parameter-free last-iterate methods applicable to important problem classes for which global smoothness is unrealistic. Our numerical experiments on bilinear matrix games, LASSO, minimax group fairness, and state-of-the-art maximum entropy sampling relaxations demonstrate wide applicability of our results as well as strong last-iterate performance and significant improvements over existing methods.