Adaptive Delayed-Update Cyclic Algorithm for Variational Inequalities

TL;DR

The paper introduces ADUCA, a parameter-free cyclic algorithm for variational inequalities that uses delayed operator information, enabling efficient parallel implementation and achieving near-optimal complexity.

Contribution

It presents ADUCA, a novel adaptive cyclic algorithm that requires no tuning and leverages delayed updates for improved parallel and distributed performance.

Findings

Achieves near-optimal global oracle complexity for variational inequalities.

Does not require line search or Lipschitz constant knowledge.

Compatible with parallel and distributed computing environments.

Abstract

Cyclic block coordinate methods are a fundamental class of first-order algorithms, widely used in practice for their simplicity and strong empirical performance. Yet, their theoretical behavior remains challenging to explain, and setting their step sizes -- beyond classical coordinate descent for minimization -- typically requires careful tuning or line-search machinery. In this work, we develop $\texttt{ADUCA}$ (Adaptive Delayed-Update Cyclic Algorithm), a cyclic algorithm addressing a broad class of Minty variational inequalities with monotone Lipschitz operators. $\texttt{ADUCA}$ is parameter-free: it requires no global or block-wise Lipschitz constants and uses no per-epoch line search, except at initialization. A key feature of the algorithm is using operator information delayed by a full cycle, which makes the algorithm compatible with parallel and distributed implementations, and attractive due to weakened synchronization requirements across blocks. We prove that $\texttt{ADUCA}$ attains (near) optimal global oracle complexity as a function of target error $\epsilon >0,$ scaling with $1/\epsilon$ for monotone operators, or with $\log^2(1/\epsilon)$ for operators that are strongly monotone.