A Variational Approach for Approximating Bayesian Networks by Edge Deletion

TL;DR

This paper introduces a variational approach for approximate inference in Bayesian networks by edge deletion, optimizing auxiliary parameters through KL-divergence to improve accuracy and relate to existing methods like IBP.

Contribution

It proposes a new criterion based on KL-divergence for selecting auxiliary parameters in edge deletion, enhancing approximation quality and providing insights into IBP and its generalizations.

Findings

The KL-divergence based method improves approximation accuracy.

The approach relates to and generalizes IBP.

Effective for inference problems with exponential complexity.

Abstract

We consider in this paper the formulation of approximate inference in Bayesian networks as a problem of exact inference on an approximate network that results from deleting edges (to reduce treewidth). We have shown in earlier work that deleting edges calls for introducing auxiliary network parameters to compensate for lost dependencies, and proposed intuitive conditions for determining these parameters. We have also shown that our method corresponds to IBP when enough edges are deleted to yield a polytree, and corresponds to some generalizations of IBP when fewer edges are deleted. In this paper, we propose a different criteria for determining auxiliary parameters based on optimizing the KL-divergence between the original and approximate networks. We discuss the relationship between the two methods for selecting parameters, shedding new light on IBP and its generalizations. We also discuss the application of our new method to approximating inference problems which are exponential in constrained treewidth, including MAP and nonmyopic value of information.