Optimal Vector Compressed Sensing Using James Stein Shrinkage

TL;DR

This paper introduces SteinSense, an optimal, scalable, and parameter-free iterative algorithm for vector compressed sensing that outperforms traditional convex optimization methods, especially in high-dimensional settings.

Contribution

The paper presents SteinSense, a novel iterative algorithm for vector compressed sensing that is provably optimal, simple to implement, and does not require tuning or training data.

Findings

SteinSense outperforms traditional methods in high-dimensional vector recovery.

The algorithm is robust and performs well on real and synthetic data.

Theoretical justification is provided based on Approximate Message Passing ideas.

Abstract

The trend in modern science and technology is to take vector measurements rather than scalars, ruthlessly scaling to ever higher dimensional vectors. For about two decades now, traditional scalar Compressed Sensing has been synonymous with a Convex Optimization based procedure called Basis Pursuit. In the vector recovery case, the natural tendency is to return to a straightforward vector extension of Basis Pursuit, also based on Convex Optimization. However, Convex Optimization is provably suboptimal, particularly when $B$ is large. In this paper, we propose SteinSense, a lightweight iterative algorithm, which is provably optimal when $B$ is large. It does not have any tuning parameter, does not need any training data, requires zero knowledge of sparsity, is embarrassingly simple to implement, and all of this makes it easily scalable to high vector dimensions. We conduct a massive volume of both real and synthetic experiments that confirm the efficacy of SteinSense, and also provide theoretical justification based on ideas from Approximate Message Passing. Fascinatingly, we discover that SteinSense is quite robust, delivering the same quality of performance on real data, and even under substantial departures from conditions under which existing theory holds.