Learning Variational Inequalities from Data: Fast Generalization Rates under Strong Monotonicity

TL;DR

This paper shows that for variational inequalities with strong monotonicity, fast learning rates similar to convex optimization can be achieved using stability-based methods, extending existing techniques to more complex problems like game equilibria.

Contribution

It introduces a straightforward approach to attain fast $ heta(1/\epsilon)$ generalization rates for variational inequalities under strong monotonicity, broadening the scope of efficient learning algorithms.

Findings

Fast $\Theta(1/\epsilon)$ rates for VIs with strong monotonicity.

Extension of stability-based generalization to complex VI problems.

Applicable to multi-player game equilibria.

Abstract

Variational inequalities (VIs) are a broad class of optimization problems encompassing machine learning problems ranging from standard convex minimization to more complex scenarios like min-max optimization and computing the equilibria of multi-player games. In convex optimization, strong convexity allows for fast statistical learning rates requiring only $\Theta(1/\epsilon)$ stochastic first-order oracle calls to find an $\epsilon$-optimal solution, rather than the standard $\Theta(1/\epsilon^2)$ calls. This note provides a simple overview of how one can similarly obtain fast $\Theta(1/\epsilon)$ rates for learning VIs that satisfy strong monotonicity, a generalization of strong convexity. Specifically, we demonstrate that standard stability-based generalization arguments for convex minimization extend directly to VIs when the domain admits a small covering, or when the operator is integrable and suboptimality is measured by potential functions; such as when finding equilibria in multi-player games.