Solving stochastic weak Minty variational inequalities without increasing batch size

TL;DR

This paper proposes stochastic extragradient algorithms for weak Minty variational inequalities that avoid increasing batch sizes, using two stepsizes with only one diminishing, ensuring convergence in nonconvex-nonconcave problems.

Contribution

It introduces a novel stochastic extragradient method with two stepsizes that requires only one oracle evaluation per iteration, improving efficiency for weak MVI problems.

Findings

Almost sure convergence of the proposed algorithms

Unified analysis covering a nonlinear generalization of primal dual hybrid gradient

Effective handling of nonconvex-nonconcave weak MVI problems

Abstract

This paper introduces a family of stochastic extragradient-type algorithms for a class of nonconvex-nonconcave problems characterized by the weak Minty variational inequality (MVI). Unlike existing results on extragradient methods in the monotone setting, employing diminishing stepsizes is no longer possible in the weak MVI setting. This has led to approaches such as increasing batch sizes per iteration which can however be prohibitively expensive. In contrast, our proposed methods involves two stepsizes and only requires one additional oracle evaluation per iteration. We show that it is possible to keep one fixed stepsize while it is only the second stepsize that is taken to be diminishing, making it interesting even in the monotone setting. Almost sure convergence is established and we provide a unified analysis for this family of schemes which contains a nonlinear generalization of the celebrated primal dual hybrid gradient algorithm.