Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling

TL;DR

This paper introduces a gradient unrolling technique to learn data-adaptive linear measurement matrices for compressed sensing, improving reconstruction accuracy by exploiting additional data structure beyond sparsity.

Contribution

It proposes a novel method that unrolls the convex $ ext{l}_1$ decoder into gradient steps, enabling data-driven learning of measurement matrices while maintaining compatibility with standard decoders.

Findings

Achieves 1.1-3x fewer measurements for accurate reconstruction.

Discovers and exploits additional data structure beyond sparsity.

Outperforms previous methods in sparse datasets and label embedding tasks.

Abstract

Linear encoding of sparse vectors is widely popular, but is commonly data-independent -- missing any possible extra (but a priori unknown) structure beyond sparsity. In this paper we present a new method to learn linear encoders that adapt to data, while still performing well with the widely used $\ell_1$ decoder. The convex $\ell_1$ decoder prevents gradient propagation as needed in standard gradient-based training. Our method is based on the insight that unrolling the convex decoder into $T$ projected subgradient steps can address this issue. Our method can be seen as a data-driven way to learn a compressed sensing measurement matrix. We compare the empirical performance of 10 algorithms over 6 sparse datasets (3 synthetic and 3 real). Our experiments show that there is indeed additional structure beyond sparsity in the real datasets; our method is able to discover it and exploit it to create excellent reconstructions with fewer measurements (by a factor of 1.1-3x) compared to the previous state-of-the-art methods. We illustrate an application of our method in learning label embeddings for extreme multi-label classification, and empirically show that our method is able to match or outperform the precision scores of SLEEC, which is one of the state-of-the-art embedding-based approaches.