Mirror-prox sliding methods for solving a class of monotone variational inequalities

TL;DR

This paper introduces mirror-prox sliding algorithms for structured monotone variational inequalities, achieving optimal iteration complexity while reducing gradient computations through component identification.

Contribution

The paper presents novel mirror-prox sliding methods that efficiently solve structured VI problems by skipping certain gradient evaluations, maintaining optimal complexity.

Findings

Achieves optimal iteration complexity for deterministic VI problems.

Reduces gradient evaluations by identifying gradient components.

Extends methods to stochastic VI problems with variance considerations.

Abstract

In this paper we propose new algorithms for solving a class of structured monotone variational inequality (VI) problems over compact feasible sets. By identifying the gradient components existing in the operator of VI, we show that it is possible to skip computations of the gradients from time to time, while still maintaining the optimal iteration complexity for solving these VI problems. Specifically, for deterministic VI problems involving the sum of the gradient of a smooth convex function $\nabla G$ and a monotone operator $H$, we propose a new algorithm, called the mirror-prox sliding method, which is able to compute an $\varepsilon$-approximate weak solution with at most $O((L/\varepsilon)^{1/2})$ evaluations of $\nabla G$ and $O((L/\varepsilon)^{1/2}+M/\varepsilon)$ evaluations of $H$, where $L$ and $M$ are Lipschitz constants of $\nabla G$ and $H$, respectively. Moreover, for the case when the operator $H$ can only be accessed through its stochastic estimators, we propose a stochastic mirror-prox sliding method that can compute a stochastic $\varepsilon$-approximate weak solution with at most $O((L/\varepsilon)^{1/2})$ evaluations of $\nabla G$ and $O((L/\varepsilon)^{1/2}+M/\varepsilon + \sigma^2/\varepsilon^2)$ samples of $H$, where $\sigma$ is the variance of the stochastic samples of $H$.