Lyapunov analysis for FISTA under strong convexity

TL;DR

This paper provides a Lyapunov-based analysis of FISTA under strong convexity, demonstrating linear convergence and optimal parameter regimes, and compares its performance with FBS and variants.

Contribution

It introduces a novel Lyapunov function for FISTA under strong convexity and shows how to optimize convergence guarantees by treating strong convexity as part of the smooth component.

Findings

FISTA with the proposed Lyapunov function converges linearly.

Treating full strong convexity as smooth yields the best theoretical guarantees.

FISTA outperforms FBS when strong convexity is properly leveraged.

Abstract

In this paper, we conduct a theoretical and numerical study of the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) under strong convexity assumptions. We propose an autonomous Lyapunov function that reflects the strong convexity of the objective function, whether it arises from the smooth or non-smooth component. This Lyapunov function decreases monotonically at a linear rate along the iterations of the algorithm for a fixed inertial parameter. Our analysis demonstrates that the best theoretical convergence guarantees for FISTA in this context are obtained when the full strong convexity is treated as if it belongs to the smooth part of the objective. Within this framework, we compare the performance of forward-backward splitting (FBS) and several FISTA variants, and find that this strategy leads FISTA to outperform all other configurations, including FBS. Moreover, we identify parameter regimes in which FBS yields better performance than FISTA when the strong convexity of the non-smooth part is not leveraged appropriately.