Linearly-Convergent FISTA Variant for Composite Optimization with Duality

TL;DR

This paper introduces C-FISTA, a new variant of FISTA that achieves global linear convergence for a wider range of composite optimization problems, outperforming existing solvers on practical models.

Contribution

The paper proposes C-FISTA, a novel FISTA variant that guarantees global linear convergence for broader composite models using Fenchel duality.

Findings

C-FISTA outperforms current first-order solvers on group Lasso and group logistic regression.

C-FISTA achieves global linear convergence for a large class of convex models.

Theoretical proof of convergence using Fenchel duality.

Abstract

Many large-scale optimization problems can be expressed as composite optimization models. Accelerated first-order methods such as the fast iterative shrinkage-thresholding algorithm (FISTA) have proven effective for numerous large composite models. In this paper, we present a new variation of FISTA, to be called C-FISTA, which obtains global linear convergence for a broader class of composite models than many of the latest FISTA variants. We demonstrate the versatility and effectiveness of C-FISTA by showing C-FISTA outperforms current first-order solvers on both group Lasso and group logistic regression models. Furthermore, we utilize Fenchel duality to prove C-FISTA provides global linear convergence for a large class of convex models without the loss of global linear convergence.