Adaptivity provably helps: information-theoretic limits on $l_0$ cost of non-adaptive sensing

TL;DR

This paper establishes information-theoretic lower bounds on the $l_0$ sensing complexity for non-adaptive strategies and demonstrates that adaptive sensing can significantly reduce this complexity.

Contribution

It introduces the $l_0$ cost metric for sensing complexity and proves lower bounds for non-adaptive sensing, highlighting the advantages of adaptive strategies.

Findings

Non-adaptive sensing requires $ heta(N \log_2 N)$ $l_0$ cost for sparse signal reconstruction.

Adaptive strategies can reduce $l_0$ cost to $ ext{O}(N)$ with the same number of measurements.

The problem relates to sphere packing in high-dimensional spaces with minimum non-zero coordinates.

Abstract

The advantages of adaptivity and feedback are of immense interest in signal processing and communication with many positive and negative results. Although it is established that adaptivity does not offer substantial reductions in minimax mean square error for a fixed number of measurements, existing results have shown several advantages of adaptivity in complexity of reconstruction, accuracy of support detection, and gain in signal-to-noise ratio, under constraints on sensing energy. Sensing energy has often been measured in terms of the Frobenius Norm of the sensing matrix. This paper uses a different metric that we call the $l_0$ cost of a sensing matrix-- to quantify the complexity of sensing. Thus sparse sensing matrices have a lower cost. We derive information-theoretic lower bounds on the $l_0$ cost that hold for any non-adaptive sensing strategy. We establish that any non-adaptive sensing strategy must incur an $l_0$ cost of $\Theta\left( N \log_2(N)\right) $ to reconstruct an $N$-dimensional, one--sparse signal when the number of measurements are limited to $\Theta\left(\log_2 (N)\right)$. In comparison, bisection-type adaptive strategies only require an $l_0$ cost of at most $\mathcal{O}(N)$ for an equal number of measurements. The problem has an interesting interpretation as a sphere packing problem in a multidimensional space, such that all the sphere centres have minimum non-zero co-ordinates. We also discuss the variation in $l_0$ cost as the number of measurements increase from $\Theta\left(\log_2 (N)\right)$ to $\Theta\left(N\right)$.