Optimal Acceleration for Minimax and Fixed-Point Problems is Not Unique

TL;DR

This paper reveals that the optimal acceleration mechanisms for minimax and fixed-point problems are not unique, introducing a family of dual algorithms with identical convergence rates but different characteristics.

Contribution

It demonstrates the non-uniqueness of optimal acceleration methods and introduces dual algorithms that match existing methods' convergence rates.

Findings

Multiple optimal acceleration algorithms exist for minimax and fixed-point problems.

New dual algorithms achieve the same worst-case rates as existing methods.

The discovery broadens the landscape of accelerated optimization techniques.

Abstract

Recently, accelerated algorithms using the anchoring mechanism for minimax optimization and fixed-point problems have been proposed, and matching complexity lower bounds establish their optimality. In this work, we present the surprising observation that the optimal acceleration mechanism in minimax optimization and fixed-point problems is not unique. Our new algorithms achieve exactly the same worst-case convergence rates as existing anchor-based methods while using materially different acceleration mechanisms. Specifically, these new algorithms are dual to the prior anchor-based accelerated methods in the sense of H-duality. This finding opens a new avenue of research on accelerated algorithms since we now have a family of methods that empirically exhibit varied characteristics while having the same optimal worst-case guarantee.