A Block Coordinate and Variance-Reduced Method for Generalized Variational Inequalities of Minty Type

TL;DR

This paper introduces a randomized block coordinate method with variance reduction for generalized variational inequalities, achieving significant complexity improvements especially when block parameters are highly nonuniform.

Contribution

It develops a novel randomized block coordinate method that improves complexity bounds for variational inequalities with monotone operators, especially under nonuniform block Lipschitz parameters.

Findings

Complexity improved by a factor of order m in certain settings.

Method applies to finite-sum operators with variance reduction.

Achieves up to a √m complexity improvement over state-of-the-art.

Abstract

Block coordinate methods have been extensively studied for minimization problems, where they come with significant complexity improvements whenever the considered problems are compatible with block decomposition and, moreover, block Lipschitz parameters are highly nonuniform. For the more general class of variational inequalities with monotone operators, essentially none of the existing methods transparently shows potential complexity benefits of using block coordinate updates in such settings. Motivated by this gap, we develop a new randomized block coordinate method and study its oracle complexity and runtime. We prove that in the setting where block Lipschitz parameters are highly nonuniform -- the main setting in which block coordinate methods lead to high complexity improvements in any of the previously studied settings -- our method can lead to complexity improvements by a factor order-$m$, where $m$ is the number of coordinate blocks. The same method further applies to the more general problem with a finite-sum operator with $m$ components, where it can be interpreted as performing variance reduction. Compared to the state of the art, the method leads to complexity improvements up to a factor $\sqrt{m},$ obtained when the component Lipschitz parameters are highly nonuniform.