Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting

TL;DR

This paper investigates the fundamental limits of recovering sparse signals in noisy high-dimensional settings, establishing conditions under which perfect recovery is theoretically possible, regardless of computational complexity.

Contribution

It derives both sufficient and necessary information-theoretic conditions for perfect sparsity recovery in noisy Gaussian measurement models, extending previous work on Lasso thresholds.

Findings

Derived sufficient conditions for asymptotically perfect recovery.

Established necessary conditions any decoder must satisfy.

Complemented previous results on Lasso thresholds.

Abstract

The problem of recovering the sparsity pattern of a fixed but unknown vector $\beta^* \in \real^p based on a set of $n$ noisy observations arises in a variety of settings, including subset selection in regression, graphical model selection, signal denoising, compressive sensing, and constructive approximation. Of interest are conditions on the model dimension $p$, the sparsity index $s$ (number of non-zero entries in $\beta^*$), and the number of observations $n$ that are necessary and/or sufficient to ensure asymptotically perfect recovery of the sparsity pattern. This paper focuses on the information-theoretic limits of sparsity recovery: in particular, for a noisy linear observation model based on measurement vectors drawn from the standard Gaussian ensemble, we derive both a set of sufficient conditions for asymptotically perfect recovery using the optimal decoder, as well as a set of necessary conditions that any decoder, regardless of its computational complexity, must satisfy for perfect recovery. This analysis of optimal decoding limits complements our previous work (ARXIV: math.ST/0605740) on sharp thresholds for sparsity recovery using the Lasso ($\ell_1$-constrained quadratic programming) with Gaussian measurement ensembles.