An Improved Last-Iterate Convergence Rate for Anchored Gradient Descent Ascent

TL;DR

This paper proves that the Anchored Gradient Descent Ascent algorithm achieves an optimal last-iterate convergence rate of O(1/t) for smooth convex-concave min-max problems, resolving an open problem.

Contribution

It establishes the first proof of the exact O(1/t) last-iterate convergence rate for the algorithm, previously unconfirmed.

Findings

Achieved an O(1/t) convergence rate for the algorithm.

Resolved an open problem in the convergence analysis of min-max algorithms.

Demonstrated the proof was autonomously discovered by an AI system.

Abstract

We analyze the last-iterate convergence of the Anchored Gradient Descent Ascent algorithm for smooth convex-concave min-max problems. While previous work established a last-iterate rate of $\mathcal{O}(1/t^{2-2p})$ for the squared gradient norm, where $p \in (1/2, 1)$, it remained an open problem whether the improved exact $\mathcal{O}(1/t)$ rate is achievable. In this work, we resolve this question in the affirmative. This result was discovered autonomously by an AI system capable of writing formal proofs in Lean. The Lean proof can be accessed at https://github.com/google-deepmind/formal-conjectures/pull/3675/commits/a13226b49fd3b897f4c409194f3bcbeb96a08515