Efficient Approximations for the Marginal Likelihood of Incomplete Data Given a Bayesian Network

TL;DR

This paper compares various asymptotic approximations for the marginal likelihood of incomplete data in Bayesian networks, evaluating their accuracy and efficiency through experiments with synthetic data.

Contribution

It provides a comparative analysis of Laplace, BIC/MDL, Draper, and Cheeseman-Stutz approximations for incomplete data likelihood in Bayesian networks.

Findings

CS measure is the most accurate approximation.

Laplace approximation is considered the most accurate among those studied.

Approximations like BIC/MDL are more efficient but less accurate.

Abstract

We discuss Bayesian methods for learning Bayesian networks when data sets are incomplete. In particular, we examine asymptotic approximations for the marginal likelihood of incomplete data given a Bayesian network. We consider the Laplace approximation and the less accurate but more efficient BIC/MDL approximation. We also consider approximations proposed by Draper (1993) and Cheeseman and Stutz (1995). These approximations are as efficient as BIC/MDL, but their accuracy has not been studied in any depth. We compare the accuracy of these approximations under the assumption that the Laplace approximation is the most accurate. In experiments using synthetic data generated from discrete naive-Bayes models having a hidden root node, we find that the CS measure is the most accurate.