Moving Anchor Extragradient Methods For Smooth Structured Minimax Problems

TL;DR

This paper proposes a moving anchor acceleration technique for extragradient methods that achieves optimal convergence rates for smooth structured minimax and saddle point problems, outperforming traditional fixed anchor approaches.

Contribution

It introduces a novel moving anchor acceleration method that generalizes existing algorithms, achieving optimal convergence rates and extending to nonconvex-nonconcave saddle point problems.

Findings

Achieves the optimal O(1/k^2) convergence rate.

Numerical results show improved constants and practical efficiency.

Proximal-point preconditioning matches theoretical optimal rates.

Abstract

This work introduces a moving anchor acceleration technique to extragradient algorithms for smooth structured minimax problems. The moving anchor is introduced as a generalization of the original algorithmic anchoring framework, i.e. the EAG method introduced in [32], in hope of further acceleration. We show that the optimal order of convergence in terms of worst-case complexity on the squared gradient, O(1/k2), is achieved by our new method (where k is the number of iterations). We have also extended our algorithm to a more general nonconvex-nonconcave class of saddle point problems using the framework of [14], which slightly generalizes [32]. We obtain similar order-optimal complexity results in this extended case. In both problem settings, numerical results illustrate the efficacy of our moving anchor algorithm variants, in particular by attaining the theoretical optimal convergence rate for first order methods, as well as suggesting a better optimized constant in the big O notation which surpasses the traditional fixed anchor methods in many cases. A proximal-point preconditioned version of our algorithms is also introduced and analyzed to match optimal theoretical convergence rates.