Expectation Consistent Approximate Inference: Generalizations and Convergence

TL;DR

This paper introduces GEC, a generalized expectation consistent inference method applicable to MAP and MMSE estimation, analyzing its fixed points, convergence, and performance against theoretical optimality predictions.

Contribution

It generalizes the EC inference method, providing a unified framework for MAP and MMSE estimation with convergence and performance analysis.

Findings

GEC converges under certain conditions.

Performance aligns with replica predictions.

Fixed points correspond to solutions of the inference problem.

Abstract

Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.