FISTA and Extensions -- Review and New Insights

TL;DR

This paper reviews FISTA, an accelerated gradient method, provides new insights into its properties, variants, and iteration complexities for convex and strongly convex problems, and discusses reformulations and their relations.

Contribution

It offers a comprehensive review of FISTA, introduces a version for convex and strongly convex problems, and analyzes its iteration complexities and reformulations.

Findings

FISTA's iteration complexity for convex problems is derived.

New reformulations of FISTA are discussed and related to existing literature.

The report extends FISTA analysis to strongly convex problems.

Abstract

The purpose of this technical report is to review the main properties of an accelerated composite gradient (ACG) method commonly referred to as the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA). In addition, we state a version of FISTA for solving both convex and strongly convex composite minimization problems and derive its iteration complexities to generate iterates satisfying various stopping criteria, including one which arises in the course of solving other composite optimization problems via inexact proximal point schemes. This report also discusses different reformulations of the convex version of FISTA and how they relate to other formulations in the literature.