Accelerated Proximal Gradient Methods in the affine-quadratic case: Strong convergence and limit identification

TL;DR

This paper investigates the convergence behavior of accelerated proximal gradient methods in the affine-quadratic setting, establishing weak and strong convergence results and clarifying the limit points of the iterates.

Contribution

It provides the first analysis of the limit behavior of APG in the affine-quadratic case, showing weak convergence to the best approximation and conditions for strong convergence.

Findings

APG iterates weakly converge to the best approximation in the solution set.

Under mild conditions, APG exhibits strong convergence.

In general, APG and PGM limits do not coincide outside the affine-quadratic setting.

Abstract

Recent works by Bot-Fadili-Nguyen (arXiv:2510.22715) and by Jang-Ryu (arXiv:2510.23513) resolve long-standing iterate convergence questions for accelerated (proximal) gradient methods. In particular, Bot-Fadili-Nguyen prove weak convergence of discrete accelerated gradient descent (AGD) iterates and, crucially, convergence of the accelerated proximal gradient (APG) method in the composite setting, with extensions to infinite-dimensional Hilbert spaces. In parallel, Jang-Ryu establish point convergence for the continuous-time accelerated flow and for discrete AGD in finite dimensions. These results leave open which minimizer is selected by the iterates. We answer this in the affine-quadratic setting: when initialized at the same point, the difference between the proximal gradient (PGM) and APG iterates converges weakly to zero. Consequently, APG converges weakly to the best approximation of the initial point in the solution set. Moreover, under mild assumptions on the parameter sequence, we obtain strong convergence of APG. The result is tight: a two-dimensional example shows that coincidence of the APG and PGM limits is specific to the affine-quadratic regime and does not hold in general.