Adaptive Path Interpolation for Sparse Systems: Application to a Simple Censored Block Model

TL;DR

This paper extends an adaptive path interpolation method to sparse systems, specifically applying it to a Censored Block Model to rigorously confirm the replica prediction for mutual information in Bayesian inference.

Contribution

The work adapts the adaptive path interpolation technique to sparse factor graphs, providing a rigorous proof of the replica prediction for a Censored Block Model.

Findings

Proves the replica prediction for the Censored Block Model.

Extends adaptive interpolation to sparse systems.

Provides a rigorous analysis of mutual information.

Abstract

Recently a new adaptive path interpolation method has been developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and generalized estimation, sparse superposition codes, or low-rank matrix and tensor estimation. For all these systems, the adaptive interpolation method directly proves that the replica symmetric prediction is exact, in a simple and unified manner. When the underlying factor graph of the inference problem is sparse the replica prediction is considerably more complicated, and rigorous results are often lacking or obtained by rather complicated methods. In this work we show how to extend the adaptive path interpolation method to sparse systems. We concentrate on a Censored Block Model, where hidden variables are measured through a binary erasure channel, for which we fully prove the replica prediction.