The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than $1/k^{2}$

TL;DR

This paper proves that a variant of Nesterov's accelerated forward-backward method converges faster than the previously established rate of 1/k^2, specifically achieving a rate of o(k^{-2}) due to a connection with differential inclusions.

Contribution

It demonstrates that a slight variant of Nesterov's method converges at a rate faster than 1/k^2, refining the theoretical understanding of its efficiency.

Findings

Convergence rate of the variant is o(k^{-2})

Connection established with second-order differential inclusion

Improves theoretical bounds on Nesterov's method

Abstract

The {\it forward-backward algorithm} is a powerful tool for solving optimization problems with a {\it additively separable} and {\it smooth} + {\it nonsmooth} structure. In the convex setting, a simple but ingenious acceleration scheme developed by Nesterov has been proved useful to improve the theoretical rate of convergence for the function values from the standard $\mathcal O(k^{-1})$ down to $\mathcal O(k^{-2})$. In this short paper, we prove that the rate of convergence of a slight variant of Nesterov's accelerated forward-backward method, which produces {\it convergent} sequences, is actually $o(k^{-2})$, rather than $\mathcal O(k^{-2})$. Our arguments rely on the connection between this algorithm and a second-order differential inclusion with vanishing damping.