A Dynamic Working Set Method for Compressed Sensing

TL;DR

This paper introduces a dynamic working set method (DWS) for solving compressed sensing problems efficiently by iteratively refining the solution and managing the working set, outperforming existing software in speed.

Contribution

The paper presents a novel DWS algorithm that adaptively manages the working set and achieves provable efficiency and accuracy guarantees for compressed sensing.

Findings

DWS reaches solutions with additive error within specified bounds.

Each solver call uses only a subset of variables proportional to the sparsity.

DWS outperforms state-of-the-art software in experimental tests.

Abstract

We propose a dynamic working set method (DWS) for the problem $\min_{\mathtt{x} \in \mathbb{R}^n} \frac{1}{2}\|\mathtt{Ax}-\mathtt{b}\|^2 + \eta\|\mathtt{x}\|_1$ that arises from compressed sensing. DWS manages the working set while iteratively calling a regression solver to generate progressively better solutions. Our experiments show that DWS is more efficient than other state-of-the-art software in the context of compressed sensing. Scale space such that $\|b\|=1$. Let $s$ be the number of non-zeros in the unknown signal. We prove that for any given $\varepsilon > 0$, DWS reaches a solution with an additive error $\varepsilon/\eta^2$ such that each call of the solver uses only $O(\frac{1}{\varepsilon}s\log s \log\frac{1}{\varepsilon})$ variables, and each intermediate solution has $O(\frac{1}{\varepsilon}s\log s\log\frac{1}{\varepsilon})$ non-zero coordinates.