Approximate Message Passing Algorithm with Universal Denoising and Gaussian Mixture Learning

TL;DR

This paper introduces a universal compressed sensing reconstruction algorithm combining AMP, context-based denoising, and Gaussian mixture density estimation, effective for unknown stationary ergodic signals.

Contribution

It presents a novel framework integrating AMP with universal denoising and Gaussian mixture learning, advancing signal reconstruction without prior input statistics knowledge.

Findings

State evolution holds for non-separable Bayesian denoisers

Proposed GM-based i.i.d. denoiser outperforms existing methods

Universal denoiser does not require bounded input assumptions

Abstract

We study compressed sensing (CS) signal reconstruction problems where an input signal is measured via matrix multiplication under additive white Gaussian noise. Our signals are assumed to be stationary and ergodic, but the input statistics are unknown; the goal is to provide reconstruction algorithms that are universal to the input statistics. We present a novel algorithmic framework that combines: (i) the approximate message passing (AMP) CS reconstruction framework, which solves the matrix channel recovery problem by iterative scalar channel denoising; (ii) a universal denoising scheme based on context quantization, which partitions the stationary ergodic signal denoising into independent and identically distributed (i.i.d.) subsequence denoising; and (iii) a density estimation approach that approximates the probability distribution of an i.i.d. sequence by fitting a Gaussian mixture (GM) model. In addition to the algorithmic framework, we provide three contributions: (i) numerical results showing that state evolution holds for non-separable Bayesian sliding-window denoisers; (ii) an i.i.d. denoiser based on a modified GM learning algorithm; and (iii) a universal denoiser that does not need information about the range where the input takes values from or require the input signal to be bounded. We provide two implementations of our universal CS recovery algorithm with one being faster and the other being more accurate. The two implementations compare favorably with existing universal reconstruction algorithms in terms of both reconstruction quality and runtime.