Another look at the fast iterative shrinkage/thresholding algorithm (FISTA)

TL;DR

This paper offers a new perspective on FISTA by linking it to worst-case bound optimization and introduces a novel algorithm based on optimizing step coefficients for better acceleration.

Contribution

It reinterprets FISTA through worst-case analysis and proposes a new accelerated method by optimizing step coefficients via Performance Estimation Problem.

Findings

FISTA is shown to be an optimized approach for worst-case bounds.

A new algorithm is proposed by optimizing step coefficients.

The new method improves acceleration over traditional proximal gradient methods.

Abstract

This paper provides a new way of developing the fast iterative shrinkage/thresholding algorithm (FISTA) that is widely used for minimizing composite convex functions with a nonsmooth term such as the $\ell_1$ regularizer. In particular, this paper shows that FISTA corresponds to an optimized approach to accelerating the proximal gradient method with respect to a worst-case bound of the cost function. This paper then proposes a new algorithm that is derived by instead optimizing the step coefficients of the proximal gradient method with respect to a worst-case bound of the composite gradient mapping. The proof is based on the worst-case analysis called Performance Estimation Problem.