Mean Field Methods for a Special Class of Belief Networks

TL;DR

This paper introduces mean-field approximation methods for a broad class of belief networks, including sigmoid and noisy-or types, using Plefka's theory and Taylor series expansions to improve computational tractability.

Contribution

It extends mean-field approximation techniques to a wider class of belief networks and proposes new computationally feasible schemes based on Plefka's theory and Taylor series.

Findings

Proposed mean-field schemes are computationally attractive.

First order approximation aligns with Saul, Jaakkola, and Jordan's approach.

Small scale experiments show promising results.

Abstract

The chief aim of this paper is to propose mean-field approximations for a broad class of Belief networks, of which sigmoid and noisy-or networks can be seen as special cases. The approximations are based on a powerful mean-field theory suggested by Plefka. We show that Saul, Jaakkola and Jordan' s approach is the first order approximation in Plefka's approach, via a variational derivation. The application of Plefka's theory to belief networks is not computationally tractable. To tackle this problem we propose new approximations based on Taylor series. Small scale experiments show that the proposed schemes are attractive.