Last-Iterate Convergence of Anchored Gradient Descent

TL;DR

This paper extends the anchored gradient descent method to monotone inclusion problems, proving a last-iterate convergence rate of O(1/√T) for the first time in this setting.

Contribution

It introduces the first analysis of anchored gradient descent for monotone inclusion problems, establishing convergence guarantees in this general framework.

Findings

Anchored gradient descent achieves O(1/√T) convergence rate.

The method extends previous unconstrained results to constrained problems.

The analysis uses techniques from recent monotone operator theory.

Abstract

We study the monotone inclusion problem $0\in F(z)+A(z)$, where $F$ is monotone and Lipschitz, and $A$ is maximally monotone, a framework that encompasses monotone variational inequalities and convex-concave saddle-point problems with constraints or regularization. It is well known that vanilla gradient descent diverges for this problem, whereas optimism-based methods such as Extragradient and accelerated methods that combine both optimism and anchoring, such as Extra Anchored Gradient, achieve last-iterate convergence. However, the anchoring-only method, anchored gradient descent, has been studied only in the unconstrained setting [RYY19, SST+26]. In this note, we extend the anchored gradient descent method to the monotone inclusion problem and prove a last-iterate convergence rate of $O(1/\sqrt{T})$ in terms of the tangent residual. We build on the recent proof in the unconstrained setting [SST+26] and use techniques from [COZ24] to extend it to the general inclusion setting.