Polarization of the Renyi Information Dimension with Applications to Compressed Sensing

TL;DR

This paper demonstrates that Hadamard matrices polarize the Renyi information dimension of i.i.d. mixture variables, enabling deterministic sensing matrix construction for compressed sensing with performance comparable to random matrices.

Contribution

It introduces the polarization of the Renyi information dimension using Hadamard matrices and applies this to design explicit sensing matrices for compressed sensing.

Findings

RID polarizes to 0 and 1, following BEC pattern.

Constructed deterministic sensing matrices perform competitively.

Extended polarization results to multi-terminal scenarios.

Abstract

In this paper, we show that the Hadamard matrix acts as an extractor over the reals of the Renyi information dimension (RID), in an analogous way to how it acts as an extractor of the discrete entropy over finite fields. More precisely, we prove that the RID of an i.i.d. sequence of mixture random variables polarizes to the extremal values of 0 and 1 (corresponding to discrete and continuous distributions) when transformed by a Hadamard matrix. Further, we prove that the polarization pattern of the RID admits a closed form expression and follows exactly the Binary Erasure Channel (BEC) polarization pattern in the discrete setting. We also extend the results from the single- to the multi-terminal setting, obtaining a Slepian-Wolf counterpart of the RID polarization. We discuss applications of the RID polarization to Compressed Sensing of i.i.d. sources. In particular, we use the RID polarization to construct a family of deterministic $\pm 1$-valued sensing matrices for Compressed Sensing. We run numerical simulations to compare the performance of the resulting matrices with that of random Gaussian and random Hadamard matrices. The results indicate that the proposed matrices afford competitive performances while being explicitly constructed.