Bayes-Optimal Estimation in Generalized Linear Models via Spatial Coupling

TL;DR

This paper demonstrates that spatially coupled sensing matrices enable AMP algorithms to achieve the theoretical minimum MSE in generalized linear models, closing the gap between feasible estimators and optimal performance.

Contribution

It introduces a spatially coupled design for GLMs and proves that AMP with this design attains the asymptotic MMSE, improving estimation accuracy.

Findings

AMP with spatial coupling approaches the MMSE in high dimensions.

Spatially coupled designs outperform i.i.d. Gaussian matrices in finite samples.

Numerical results confirm substantial MSE reduction in phase retrieval and regression.

Abstract

We consider the problem of signal estimation in a generalized linear model (GLM). GLMs include many canonical problems in statistical estimation, such as linear regression, phase retrieval, and 1-bit compressed sensing. Recent work has precisely characterized the asymptotic minimum mean-squared error (MMSE) for GLMs with i.i.d. Gaussian sensing matrices. However, in many models there is a significant gap between the MMSE and the performance of the best known feasible estimators. In this work, we address this issue by considering GLMs defined via spatially coupled sensing matrices. We propose an efficient approximate message passing (AMP) algorithm for estimation and prove that with a simple choice of spatially coupled design, the MSE of a carefully tuned AMP estimator approaches the asymptotic MMSE in the high-dimensional limit. To prove the result, we first rigorously characterize the asymptotic performance of AMP for a GLM with a generic spatially coupled design. This characterization is in terms of a deterministic recursion (`state evolution') that depends on the parameters defining the spatial coupling. Then, using a simple spatially coupled design and a judicious choice of functions for the AMP algorithm, we analyze the fixed points of the resulting state evolution and show that it achieves the asymptotic MMSE. Numerical results for phase retrieval and rectified linear regression show that spatially coupled designs can yield substantially lower MSE than i.i.d. Gaussian designs at finite dimensions when used with AMP algorithms.