Universal methods for variational inequalities: deterministic and stochastic cases

TL;DR

This paper introduces universal proximal mirror methods that adapt to both noise levels and Holder continuity in solving variational inequalities, achieving optimal complexity without prior knowledge of problem parameters.

Contribution

The paper presents new universal algorithms for variational inequalities that automatically adapt to problem smoothness and noise, with proven optimal complexity bounds.

Findings

Algorithms adapt to Holder continuity and noise without prior knowledge.

Achieve the lowest possible worst-case complexity for the class.

Validated effectiveness through comparison with popular image classification optimizers.

Abstract

In this paper, we propose universal proximal mirror methods to solve the variational inequality problem with Holder continuous operators in both deterministic and stochastic settings. The proposed methods automatically adapt not only to the oracle's noise (in the stochastic setting of the problem) but also to the Holder continuity of the operator without having prior knowledge of either the problem class or the nature of the operator information. We analyzed the proposed algorithms in both deterministic and stochastic settings and obtained estimates for the required number of iterations to achieve a given quality of a solution to the variational inequality. We showed that, without knowing the Holder exponent and Holder constant of the operators, the proposed algorithms have the least possible in the worst case sense complexity for the considered class of variational inequalities. We also compared the resulting stochastic algorithm with other popular optimizers for the task of image classification.