Mini-Extragradient Methods

TL;DR

This paper introduces Mini-Extragradient methods that adapt stepsize selection and reduce computational costs by focusing on dominant components and sampling, achieving faster convergence and significant speedups over classical methods.

Contribution

The paper proposes novel Mini-Extragradient algorithms with adaptive stepsize and coordinate sampling, improving efficiency and convergence guarantees for solving monotone nonlinear equations.

Findings

Achieves over 13x speedup in experiments

Provides convergence guarantees and rate analyses

Outperforms classical EG in standard applications

Abstract

The Extragradient (EG) method stands as a cornerstone algorithm for solving monotone nonlinear equations but faces two important unresolved challenges: (i) how to select stepsizes without relying on the global Lipschitz constant or expensive line-search procedures, and (ii) how to reduce the two full evaluations of the mapping required per iteration to effectively one, without compromising convergence guarantees or computational efficiency. To address the first challenge, we propose the Greedy Mini-Extragradient (Mini-EG) method, which updates only the coordinate associated with the dominant component of the mapping at each extragradient step. This design capitalizes on componentwise Lipschitz constants that are far easier to estimate than the classical global Lipschitz constant. To further lower computational cost, we introduce a Random Mini-EG variant that replaces full mapping evaluations by sampling only a single coordinate per extragradient step. Although this resolves the second challenge from a theoretical standpoint, its practical efficiency remains limited. To bridge this gap, we develop the Watchdog-Max strategy, motivated by the slow decay of dominant component magnitudes. Instead of evaluating the full mapping, Watchdog-Max identifies and tracks only two coordinates at each extragradient step, dramatically reducing per-iteration cost while retaining strong practical performance. We establish convergence guarantees and rate analyses for all proposed methods. In particular, Greedy Mini-EG achieves enhanced convergence rates that surpass the classical guarantees of the vanilla EG method in several standard application settings. Numerical experiments on regularized decentralized logistic regression and compressed sensing show speedups exceeding $13\times$ compared with the classical EG method on both synthetic and real datasets.