Accelerated Minimax Algorithms Flock Together

TL;DR

This paper introduces the merging path property of accelerated minimax algorithms, proving their convergence and developing new algorithms with improved performance in strongly-convex-strongly-concave and prox-grad settings.

Contribution

It reveals the merging path property of accelerated minimax algorithms and leverages it to establish convergence and create new state-of-the-art algorithms.

Findings

Algorithms exhibit rapid merging trajectories

Proven convergence of existing accelerated minimax methods

New algorithms outperform previous methods in key setups

Abstract

Several new accelerated methods in minimax optimization and fixed-point iterations have recently been discovered, and, interestingly, they rely on a mechanism distinct from Nesterov's momentum-based acceleration. In this work, we show that these accelerated algorithms exhibit what we call the merging path (MP) property; the trajectories of these algorithms merge quickly. Using this novel MP property, we establish point convergence of existing accelerated minimax algorithms and derive new state-of-the-art algorithms for the strongly-convex-strongly-concave setup and for the prox-grad setup.