Optimal Methods for Higher-Order Smooth Monotone Variational Inequalities

TL;DR

This paper introduces the first line search-free, optimal algorithms for higher-order smooth monotone variational inequalities, extending prior work and establishing the fundamental limits of their complexity.

Contribution

The paper presents a novel $p^{th}$-order method for smooth MVIs that achieves optimal convergence rates without line search, applicable in constrained and non-Euclidean settings.

Findings

Achieves $O( ilde{ ext{O}}(rac{1}{ ext{epsilon}^{2/(p+1)}}))$ rate without line search.

Provides the first lower bounds for smooth MVIs for $p > 1$.

Establishes a separation between solving smooth MVIs and convex optimization.

Abstract

In this work, we present new simple and optimal algorithms for solving the variational inequality (VI) problem for $p^{th}$-order smooth, monotone operators -- a problem that generalizes convex optimization and saddle-point problems. Recent works (Bullins and Lai (2020), Lin and Jordan (2021), Jiang and Mokhtari (2022)) present methods that achieve a rate of $\tilde{O}(\epsilon^{-2/(p+1)})$ for $p\geq 1$, extending results by (Nemirovski (2004)) and (Monteiro and Svaiter (2012)) for $p=1,2$. A drawback to these approaches, however, is their reliance on a line search scheme. We provide the first $p^{\textrm{th}}$-order method that achieves a rate of $O(\epsilon^{-2/(p+1)}).$ Our method does not rely on a line search routine, thereby improving upon previous rates by a logarithmic factor. Building on the Mirror Prox method of Nemirovski (2004), our algorithm works even in the constrained, non-Euclidean setting. Furthermore, we prove the optimality of our algorithm by constructing matching lower bounds. These are the first lower bounds for smooth MVIs beyond convex optimization for $p > 1$. This establishes a separation between solving smooth MVIs and smooth convex optimization, and settles the oracle complexity of solving $p^{\textrm{th}}$-order smooth MVIs.