Divergence Estimation in Message Passing algorithms

TL;DR

This paper introduces two new divergence estimators for message passing algorithms that eliminate the need for extra denoiser executions, maintaining accuracy while reducing computational costs in compressed sensing applications.

Contribution

The paper develops two large system limit models for Onsager correction divergence estimation, enabling efficient and accurate divergence estimation without additional denoiser runs.

Findings

Proposed divergence estimators match or outperform BB-MC accuracy.

Estimators significantly reduce computational cost in SMP algorithms.

Models are validated through theoretical analysis and simulations.

Abstract

Many modern imaging applications can be modeled as compressed sensing linear inverse problems. When the measurement operator involved in the inverse problem is sufficiently random, denoising Scalable Message Passing (SMP) algorithms have a potential to demonstrate high efficiency in recovering compressed data. One of the key components enabling SMP to achieve fast convergence, stability and predictable dynamics is the Onsager correction that must be updated at each iteration of the algorithm. This correction involves the denoiser's divergence that is traditionally estimated via the Black-Box Monte Carlo (BB-MC) method \cite{MC-divergence}. While the BB-MC method demonstrates satisfying accuracy of estimation, it requires executing the denoiser additional times at each iteration and might lead to a substantial increase in computational cost of the SMP algorithms. In this work we develop two Large System Limit models of the Onsager correction for denoisers operating within SMP algorithms and use these models to propose two practical classes of divergence estimators that require no additional executions of the denoiser and demonstrate similar or superior correction compared to the BB-MC method.