Towards Designing Optimal Sensing Matrices for Generalized Linear Inverse Problems

TL;DR

This paper investigates how the spectral properties of sensing matrices affect the performance of the expectation propagation algorithm in nonlinear inverse problems, providing insights for optimal matrix design.

Contribution

It introduces a spectral spikiness measure and analyzes its impact on EP performance across different nonlinear inverse problems, unifying and generalizing previous findings.

Findings

Spikier spectra can improve or impair recovery depending on the nonlinear function.

In phase-retrieval, spikier spectra are advantageous for EP.

In 1-bit compressed sensing, less spiky spectra yield better results.

Abstract

We consider an inverse problem $\mathbf{y}= f(\mathbf{Ax})$, where $\mathbf{x}\in\mathbb{R}^n$ is the signal of interest, $\mathbf{A}$ is the sensing matrix, $f$ is a nonlinear function and $\mathbf{y} \in \mathbb{R}^m$ is the measurement vector. In many applications, we have some level of freedom to design the sensing matrix $\mathbf{A}$, and in such circumstances we could optimize $\mathbf{A}$ to achieve better reconstruction performance. As a first step towards optimal design, it is important to understand the impact of the sensing matrix on the difficulty of recovering $\mathbf{x}$ from $\mathbf{y}$. In this paper, we study the performance of one of the most successful recovery methods, i.e., the expectation propagation (EP) algorithm. We define a notion of spikiness for the spectrum of $\bmmathbfA}$ and show the importance of this measure for the performance of EP. We show that whether a spikier spectrum can hurt or help the recovery performance depends on $f$. Based on our framework, we are able to show that, in phase-retrieval problems, matrices with spikier spectrums are better for EP, while in 1-bit compressed sensing problems, less spiky spectrums lead to better performance. Our results unify and substantially generalize existing results that compare Gaussian and orthogonal matrices, and provide a platform towards designing optimal sensing systems.