Linear convergence of iterative soft-thresholding

TL;DR

This paper presents a unified analysis demonstrating that iterative soft-thresholding algorithms converge linearly under certain conditions related to operator properties and sparsity patterns, with explicit constants for specific cases.

Contribution

The paper introduces a new convergence analysis framework for iterative soft-thresholding, establishing linear convergence under finite basis injectivity or strict sparsity, and extends techniques to related algorithms.

Findings

Convergence is linear when the operator has finite basis injectivity.

Explicit constants for convergence are derived for compact operators.

The analysis applies to related methods like joint sparsity thresholding and accelerated gradient projection.

Abstract

In this article a unified approach to iterative soft-thresholding algorithms for the solution of linear operator equations in infinite dimensional Hilbert spaces is presented. We formulate the algorithm in the framework of generalized gradient methods and present a new convergence analysis. As main result we show that the algorithm converges with linear rate as soon as the underlying operator satisfies the so-called finite basis injectivity property or the minimizer possesses a so-called strict sparsity pattern. Moreover it is shown that the constants can be calculated explicitly in special cases (i.e. for compact operators). Furthermore, the techniques also can be used to establish linear convergence for related methods such as the iterative thresholding algorithm for joint sparsity and the accelerated gradient projection method.