The iterates of FISTA convergence even under inexact computations and stochastic gradients

TL;DR

This paper proves that FISTA's iterates converge in infinite-dimensional Hilbert spaces even with inexact and stochastic computations, extending recent convergence results to more practical scenarios.

Contribution

It extends convergence proofs of FISTA to infinite-dimensional spaces with inexact and stochastic gradient and proximity computations.

Findings

FISTA iterates converge weakly in infinite-dimensional spaces.

Convergence holds despite inexact and stochastic gradient computations.

Results generalize recent finite-dimensional convergence proofs.

Abstract

Very recently, the papers "Point Convergence of Nesterov's Accelerated Gradient Method: An AI-Assisted Proof" by Jang and Ryu, and "The Iterates of Nesterov's Accelerated Algorithm Converge in the Critical Regimes" by Bot, Fadili, and Nguyen simultaneously have resolved a long-standing open problem concerning Nesterov's accelerated gradient method. These works show that the iterates of the algorithm (known in its composite form as FISTA) indeed converge to an optimal solution. In this work, we extend these results and prove that, in infinite dimensional Hilbert spaces, the iterates of such an algorithm still converge (in the weak sense) even when the proximity operator and the gradient are computed inexactly, with the latter possibly stochastic.