A Fast Optimistic Method for Monotone Variational Inequalities

TL;DR

This paper introduces fOGDA-VI, a fast and efficient algorithm for solving monotone variational inequalities with strong theoretical guarantees and promising empirical results, including applications to generative adversarial networks.

Contribution

The paper presents a novel algorithm, fOGDA-VI, that achieves the best known convergence rates for monotone variational inequalities using minimal gradient and projection evaluations per iteration.

Findings

Achieves $o(1/k)$ convergence rate in restricted gap and residual.

Provides convergence guarantees for the sequence of iterates.

Demonstrates promising empirical results on matrix games and GAN training.

Abstract

We study monotone variational inequalities that can arise as optimality conditions for constrained convex optimisation or convex-concave minimax problems and propose a novel algorithm that uses only one gradient/operator evaluation and one projection onto the constraint set per iteration. The algorithm, which we call fOGDA-VI, achieves a $o \left( \frac{1}{k} \right)$ rate of convergence in terms of the restricted gap function as well as the natural residual for the last iterate. Moreover, we provide a convergence guarantee for the sequence of iterates to a solution of the variational inequality. These are the best theoretical convergence results for numerical methods for (only) monotone variational inequalities reported in the literature. To empirically validate our algorithm we investigate a two-player matrix game with mixed strategies of the two players. Concluding, we show promising results regarding the application of fOGDA-VI to the training of generative adversarial nets.