On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty

TL;DR

This paper introduces a generalized iterative soft-thresholding algorithm for non-separable $ ext{l}_1$ penalized least squares problems, with proven convergence and applicability to various convex penalties.

Contribution

It extends the traditional iterative soft-thresholding algorithm to handle non-separable penalties and provides convergence guarantees.

Findings

Algorithm requires four matrix-vector multiplications per iteration.

Convergence is proven with a 1/N rate for the functional.

Special cases recover traditional and dual gradient projection algorithms.

Abstract

An explicit algorithm for the minimization of an $\ell_1$ penalized least squares functional, with non-separable $\ell_1$ term, is proposed. Each step in the iterative algorithm requires four matrix vector multiplications and a single simple projection on a convex set (or equivalently thresholding). Convergence is proven and a 1/N convergence rate is derived for the functional. In the special case where the matrix in the $\ell_1$ term is the identity (or orthogonal), the algorithm reduces to the traditional iterative soft-thresholding algorithm. In the special case where the matrix in the quadratic term is the identity (or orthogonal), the algorithm reduces to a gradient projection algorithm for the dual problem. By replacing the projection with a simple proximity operator, other convex non-separable penalties than those based on an $\ell_1$-norm can be handled as well.