On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty

TL;DR

This paper analyzes the convergence of ISTA when applied to a cost function with a weakly convex penalty, showing that large stepsizes can improve convergence speed in this setting.

Contribution

It generalizes convergence results of ISTA to weakly convex penalties and demonstrates practical acceleration with larger stepsizes.

Findings

Large stepsizes can accelerate ISTA convergence.

Convergence results extend to non-convex penalties with convex overall cost.

Experimental validation shows improved convergence speed.

Abstract

We consider the iterative shrinkage/thresholding algorithm (ISTA) applied to a cost function composed of a data fidelity term and a penalty term. The penalty is non-convex but the concavity of the penalty is accounted for by the data fidelity term so that the overall cost function is convex. We provide a generalization of the convergence result for ISTA viewed as a forward-backward splitting algorithm. We also demonstrate experimentally that for the current setup, using large stepsizes in ISTA can accelerate convergence more than existing schemes proposed for the convex case, like TwIST or FISTA.