Score-Based VAMP with Fisher-Information-Based Onsager Correction

TL;DR

This paper introduces SC-VAMP, a score-based variant of VAMP that uses Fisher information for Onsager correction, enabling Jacobian-free, learned, and black-box inference in complex settings.

Contribution

The paper presents a novel score-based VAMP method that computes Onsager correction via Fisher information, extending applicability to black-box and structured sensing scenarios.

Findings

Enables Jacobian-free implementation of VAMP using Fisher information.

Extends VAMP to complex, intractable inference problems with learned score functions.

Provides an information-theoretic perspective on the Gaussian approximation in SE.

Abstract

We propose score-based VAMP (SC-VAMP), a variant of vector approximate message passing (VAMP) in which the Onsager correction is expressed and computed via conditional Fisher information, thereby enabling a Jacobian-free implementation. Using learned score functions, SC-VAMP constructs nonlinear MMSE estimators through Tweedie's formula and derives the corresponding Onsager terms from the score-norm statistics, avoiding the need for analytical derivatives of the prior or likelihood. When combined with random orthogonal/unitary mixing to mitigate non-ideal, structured or correlated sensing settings, the proposed framework extends VAMP to complex black-box inference problems where explicit modeling is intractable. Finally, by leveraging the entropic CLT, we provide an information-theoretic perspective on the Gaussian approximation underlying SE, offering insight into the decoupling principle beyond idealized i.i.d. settings, including nonlinear regimes.