Extragradient Method for $(L_0, L_1)$-Lipschitz Root-finding Problems

TL;DR

This paper extends the extragradient method to $(L_0, L_1)$-Lipschitz root-finding problems, providing new convergence guarantees and demonstrating its effectiveness through experiments in modern machine learning contexts.

Contribution

It introduces a novel step size strategy for the extragradient method under the relaxed $(L_0, L_1)$-Lipschitz condition, with proven convergence rates and local guarantees.

Findings

Sublinear convergence for monotone operators

Linear convergence for strongly monotone operators

Experimental validation of step size effectiveness

Abstract

Introduced by Korpelevich in 1976, the extragradient method (EG) has become a cornerstone technique for solving min-max optimization, root-finding problems, and variational inequalities (VIs). Despite its longstanding presence and significant attention within the optimization community, most works focusing on understanding its convergence guarantees assume the strong L-Lipschitz condition. In this work, building on the proposed assumptions by Zhang et al. [2024b] for minimization and Vankov et al.[2024] for VIs, we focus on the more relaxed $\alpha$-symmetric $(L_0, L_1)$-Lipschitz condition. This condition generalizes the standard Lipschitz assumption by allowing the Lipschitz constant to scale with the operator norm, providing a more refined characterization of problem structures in modern machine learning. Under the $\alpha$-symmetric $(L_0, L_1)$-Lipschitz condition, we propose a novel step size strategy for EG to solve root-finding problems and establish sublinear convergence rates for monotone operators and linear convergence rates for strongly monotone operators. Additionally, we prove local convergence guarantees for weak Minty operators. We supplement our analysis with experiments validating our theory and demonstrating the effectiveness and robustness of the proposed step sizes for EG.