Communications-Inspired Projection Design with Application to Compressive Sensing

TL;DR

This paper introduces a novel projection design method inspired by communications theory to enhance signal recovery in compressive sensing, demonstrating improved performance over traditional methods and enabling fast online adaptation.

Contribution

It extends information-theoretic projection design to the Renyi entropy domain and applies it to compressive sensing, showing significant performance gains and fast online implementation.

Findings

Improved image recovery in compressive sensing using the new projection design.

Demonstrated the effectiveness of mode exposure and mode alignment operations.

Achieved state-of-the-art adaptive signal recovery with fast online projections.

Abstract

We consider the recovery of an underlying signal x \in C^m based on projection measurements of the form y=Mx+w, where y \in C^l and w is measurement noise; we are interested in the case l < m. It is assumed that the signal model p(x) is known, and w CN(w;0,S_w), for known S_W. The objective is to design a projection matrix M \in C^(l x m) to maximize key information-theoretic quantities with operational significance, including the mutual information between the signal and the projections I(x;y) or the Renyi entropy of the projections h_a(y) (Shannon entropy is a special case). By capitalizing on explicit characterizations of the gradients of the information measures with respect to the projections matrix, where we also partially extend the well-known results of Palomar and Verdu from the mutual information to the Renyi entropy domain, we unveil the key operations carried out by the optimal projections designs: mode exposure and mode alignment. Experiments are considered for the case of compressive sensing (CS) applied to imagery. In this context, we provide a demonstration of the performance improvement possible through the application of the novel projection designs in relation to conventional ones, as well as justification for a fast online projections design method with which state-of-the-art adaptive CS signal recovery is achieved.