Sparse Signal Recovery using Generalized Approximate Message Passing with Built-in Parameter Estimation

TL;DR

This paper introduces PE-GAMP, an extended GAMP algorithm with built-in parameter estimation that jointly recovers sparse signals and estimates unknown distribution parameters, outperforming EM-based methods especially in noisy and complex scenarios.

Contribution

The paper proposes PE-GAMP, a novel extension of GAMP that incorporates joint parameter estimation with sparse signal recovery, enabling more robust and flexible modeling.

Findings

PE-GAMP matches oracle GAMP in noiseless recovery.

PE-GAMP outperforms EM-based methods in noisy conditions.

PE-GAMP enables non-negative sparse coding for image classification.

Abstract

The generalized approximate message passing (GAMP) algorithm under the Bayesian setting shows advantage in recovering under-sampled sparse signals from corrupted observations. Compared to conventional convex optimization methods, it has a much lower complexity and is computationally tractable. In the GAMP framework, the sparse signal and the observation are viewed to be generated according to some pre-specified probability distributions in the input and output channels. However, the parameters of the distributions are usually unknown in practice. In this paper, we propose an extended GAMP algorithm with built-in parameter estimation (PE-GAMP) and present its empirical convergence analysis. PE-GAMP treats the parameters as unknown random variables with simple priors and jointly estimates them with the sparse signals. Compared with Expectation Maximization (EM) based parameter estimation methods, the proposed PE-GAMP could draw information from the prior distributions of the parameters to perform parameter estimation. It is also more robust and much simpler, which enables us to consider more complex signal distributions apart from the usual Bernoulli-Gaussian (BGm) mixture distribution. Specifically, the formulations of Bernoulli-Exponential mixture (BEm) distribution and Laplace distribution are given in this paper. Simulated noiseless sparse signal recovery experiments demonstrate that the performance of the proposed PE-GAMP matches the oracle GAMP algorithm. When noise is present, both the simulated experiments and the real image recovery experiments show that PE-GAMP is still able to maintain its robustness and outperform EM based parameter estimation method when the sampling ratio is small. Additionally, using the BEm formulation of the PE-GAMP, we can successfully perform non-negative sparse coding of local image patches and provide useful features for the image classification task.