Methods for Solving Variational Inequalities with Markovian Stochasticity

TL;DR

This paper introduces a new stochastic method using the Extragradient technique to solve variational inequalities affected by Markovian noise, with proven convergence and experimental insights.

Contribution

The paper proposes a novel Extragradient-based stochastic algorithm specifically designed for variational inequalities with Markovian stochasticity, supported by theoretical convergence analysis.

Findings

Proven convergence under mild assumptions including Lipschitzness and strong monotonicity.

Experimental results show the impact of Markov process mixing time on convergence.

Method effectively handles Markovian noise in VI problems.

Abstract

In this paper, we present a novel stochastic method for solving variational inequalities (VI) in the context of Markovian noise. By leveraging Extragradient technique, we can productively solve VI optimization problems characterized by Markovian dynamics. We demonstrate the efficacy of proposed method through rigorous theoretical analysis, proving convergence under quite mild assumptions of $L$-Lipschitzness, strong monotonicity of the operator and boundness of the noise only at the optimum. In order to gain further insight into the nature of Markov processes, we conduct the experiments to investigate the impact of the mixing time parameter on the convergence of the algorithm.