Single-loop variance reduction methods in Bregman setups for finite-sum structured variational inequalities

TL;DR

This paper introduces a novel single-loop variance-reduced algorithm for finite-sum variational inequalities using Bregman distances, achieving optimal complexity and linear convergence in certain settings.

Contribution

It is the first to establish linear convergence for finite-sum VI in Bregman setups and improves existing complexity bounds with a new variance reduction method.

Findings

Achieves optimal complexity of O(√M/ε) for monotone VI

Derives complexity of O(1/ε²) for non-monotone VI

Demonstrates effectiveness through numerical experiments

Abstract

In this paper, we address variational inequalities (VI) with a finite sum structure by proposing a novel single-loop variance-reduced algorithm that incorporates the Bregman distance. Under the monotone setting, we establish the almost sure convergence of the proposed algorithm and prove that it achieves the optimal complexity of $\mathcal{O}\left(\frac{\sqrt{M}}{\varepsilon }\right)$ for finding an $\varepsilon$-gap. Furthermore, under the non-monotone setting, we derive a complexity of $\mathcal{O}\left(\frac{1}{\varepsilon^2 }\right)$ of the algorithm. Our proposed method yields complexity results that either match or improve the state-of-the-art complexity bounds reported in existing literature. Notably, this work is the first to rigorously establish the linear convergence rate of the algorithm for solving finite-sum variational inequalities in Bregman setups. Finally, we report two numerical experiments to validate the effectiveness and practical performance of our method.