Dependent Dirichlet Priors and Optimal Linear Estimators for Belief Net Parameters

TL;DR

This paper introduces dependent Dirichlet priors for Bayesian belief networks, allowing for correlated parameter rows, and develops optimal linear estimators that improve efficiency by sharing information among similar conditioning events.

Contribution

It proposes a flexible family of dependent Dirichlet priors and simple methods for optimal linear estimation of CP-table parameters, enhancing estimation efficiency.

Findings

Dependent Dirichlet priors capture correlations among CP-table rows.

Optimal linear estimators outperform traditional methods with small sample sizes.

Sharing information among similar rows improves estimation accuracy.

Abstract

A Bayesian belief network is a model of a joint distribution over a finite set of variables, with a DAG structure representing immediate dependencies among the variables. For each node, a table of parameters (CPtable) represents local conditional probabilities, with rows indexed by conditioning events (assignments to parents). CP-table rows are usually modeled as independent random vectors, each assigned a Dirichlet prior distribution. The assumption that rows are independent permits a relatively simple analysis but may not reflect actual prior opinion about the parameters. Rows representing similar conditioning events often have similar conditional probabilities. This paper introduces a more flexible family of "dependent Dirichlet" prior distributions, where rows are not necessarily independent. Simple methods are developed to approximate the Bayes estimators of CP-table parameters with optimal linear estimators; i.e., linear combinations of sample proportions and prior means. This approach yields more efficient estimators by sharing information among rows. Improvements in efficiency can be substantial when a CP-table has many rows and samples sizes are small.