Stochastic Variance-Reduced Forward-Reflected-Backward Splitting Methods for Nonmonotone Generalized Equations

TL;DR

This paper introduces two innovative stochastic variance-reduction algorithms for solving nonmonotone generalized equations, achieving state-of-the-art complexity and demonstrating improved performance over existing methods.

Contribution

The paper presents novel stochastic variance-reduction algorithms combining forward-reflected-backward splitting with new estimators inspired by SVRG and SAGA, tailored for nonmonotone problems.

Findings

Achieve oracle complexity of O(n + n^{2/3}ε^{-2}) for ε-solutions.

Algorithms outperform existing methods in numerical tests.

New estimators differ significantly from prior approaches in minimax and variational inequality problems.

Abstract

We develop two novel stochastic variance-reduction methods to approximate solutions of a class of nonmonotone [generalized] equations. Our algorithms leverage a new combination of ideas from the forward-reflected-backward splitting method and a class of unbiased variance-reduced estimators. We construct two new stochastic estimators within this class, inspired by the well-known SVRG and SAGA estimators. These estimators significantly differ from existing approaches used in minimax and variational inequality problems. By appropriately choosing parameters, both algorithms achieve a state-of-the-art oracle complexity of $\mathcal{O}(n + n^{2/3}\epsilon^{-2})$ for obtaining an $\epsilon$-solution in terms of the operator residual norm for a class of nonmonotone problems, where $n$ is the number of summands and $\epsilon$ signifies the desired accuracy. This complexity aligns with the best-known results in SVRG and SAGA methods for stochastic nonconvex optimization. We test our algorithms on some numerical examples and compare them with existing methods. The results demonstrate promising improvements offered by the new methods compared to their competitors.