Gradient-based methods for sparse recovery

TL;DR

This paper analyzes the convergence of the SpaSRA algorithm for sparse recovery, showing sublinear and linear rates under convexity assumptions, and introduces an improved version with adaptive line search tested on signal and image reconstruction tasks.

Contribution

The paper provides convergence analysis for SpaSRA, including an improved algorithm with adaptive line search, applied to signal processing and image reconstruction.

Findings

Objective error decreases as 1/(b+k) for convex f.

Linear convergence rate for strongly convex f.

Enhanced algorithm with cycle BB iteration and adaptive line search.

Abstract

The convergence rate is analyzed for the SpaSRA algorithm (Sparse Reconstruction by Separable Approximation) for minimizing a sum $f (\m{x}) + \psi (\m{x})$ where $f$ is smooth and $\psi$ is convex, but possibly nonsmooth. It is shown that if $f$ is convex, then the error in the objective function at iteration $k$, for $k$ sufficiently large, is bounded by $a/(b+k)$ for suitable choices of $a$ and $b$. Moreover, if the objective function is strongly convex, then the convergence is $R$-linear. An improved version of the algorithm based on a cycle version of the BB iteration and an adaptive line search is given. The performance of the algorithm is investigated using applications in the areas of signal processing and image reconstruction.