Some Adaptive First-order Methods for Variational Inequalities with Relatively Strongly Monotone Operators and Generalized Smoothness

TL;DR

This paper introduces adaptive algorithms for variational inequalities with relatively strongly monotone operators, avoiding restart techniques and achieving convergence rates comparable to accelerated methods, supported by theoretical analysis and numerical experiments.

Contribution

The paper proposes novel adaptive methods for variational inequalities that do not rely on restart techniques, with theoretical convergence guarantees and practical effectiveness.

Findings

Algorithms achieve convergence rates similar to accelerated methods.

Numerical experiments demonstrate the effectiveness of the proposed algorithms.

Avoidance of restart techniques simplifies implementation in applied problems.

Abstract

In this paper, we introduce some adaptive methods for solving variational inequalities with relatively strongly monotone operators. Firstly, we focus on the modification of the recently proposed, in smooth case [1], adaptive numerical method for generalized smooth (with H\"older condition) saddle point problem, which has convergence rate estimates similar to accelerated methods. We provide the motivation for such an approach and obtain theoretical results of the proposed method. Our second focus is the adaptation of widespread recently proposed methods for solving variational inequalities with relatively strongly monotone operators. The key idea in our approach is the refusal of the well-known restart technique, which in some cases causes difficulties in implementing such algorithms for applied problems. Nevertheless, our algorithms show a comparable rate of convergence with respect to algorithms based on the above-mentioned restart technique. Also, we present some numerical experiments, which demonstrate the effectiveness of the proposed methods. [1] Jin, Y., Sidford, A., & Tian, K. (2022). Sharper rates for separable minimax and finite sum optimization via primal-dual extragradient methods. arXiv preprint arXiv:2202.04640.