Adaptive sensing using deterministic partial Hadamard matrices

TL;DR

This paper explores how deterministic partial Hadamard matrices can preserve the entropy of discrete random vectors, highlighting their potential for efficient sensing and reconstruction, while showing limitations for continuous vectors.

Contribution

It demonstrates that deterministic partial Hadamard matrices can preserve entropy for discrete vectors with vanishing row fraction, introducing a new entropy power inequality and polar code martingale approach.

Findings

Deterministic partial Hadamard matrices preserve entropy for i.i.d. discrete vectors.

Such matrices cannot preserve entropy for i.i.d. continuous vectors.

The approach uses a novel entropy power inequality and polar code martingale techniques.

Abstract

This paper investigates the construction of deterministic matrices preserving the entropy of random vectors with a given probability distribution. In particular, it is shown that for random vectors having i.i.d. discrete components, this is achieved by selecting a subset of rows of a Hadamard matrix such that (i) the selection is deterministic (ii) the fraction of selected rows is vanishing. In contrast, it is shown that for random vectors with i.i.d. continuous components, no partial Hadamard matrix of reduced dimension allows to preserve the entropy. These results are in agreement with the results of Wu-Verdu on almost lossless analog compression. This paper is however motivated by the complexity attribute of Hadamard matrices, which allows the use of efficient and stable reconstruction algorithms. The proof technique is based on a polar code martingale argument and on a new entropy power inequality for integer-valued random variables.