Beyond First-Order Methods for $\ell_p$-Structured Non-Monotone Variational Inequalities

TL;DR

This paper introduces high-order algorithms for $ abla_p$-structured non-monotone variational inequalities, enabling convergence to $ ext{l}_p$-norm stationary points for a broader range of parameters, surpassing previous limitations.

Contribution

It develops high-order methods that achieve convergence to $ ext{l}_p$-norm stationary points for $ ho > 0$, extending prior work limited to $ ho=0$, and generalizes results to monotone operators for $p eq 2$.

Findings

Algorithms converge to $ ext{l}_p$-stationary points for $ ho > 0$

Extended convergence results to monotone operators for $p eq 2$

Techniques adapted to continuous-time settings

Abstract

We propose novel high-order algorithms for a class of $\ell_p$-structured non-monotone variational inequalities. In particular, work by Diakonikolas et al. (2021), which introduced the weak Minty variational inequality (weak-MVI) setting, showed how to find an approximate first-order Euclidean stationary point for a strictly positive range of the weak-MVI parameter $\rho$. However, for the $\ell_p$-norm stationary point setting ($p \neq 2$), their guarantees are limited to $\rho=0$, which recovers the standard MVI setting. In this work, we address this gap by presenting a suite of high-order methods that converge to $\ell_p$-norm stationary points for a suitable range of $\rho > 0$, thereby circumventing previous fundamental challenges in $\ell_p$ settings. We further show convergence for high-order smooth \textit{monotone} operators, generalizing Adil et al. (2022) to the case where $p \geq 2$, and we extend our Euclidean techniques to continuous-time settings.