Entropy and Belief Networks

TL;DR

This paper explores the entropy properties of belief networks, demonstrating that certain models can nearly guarantee maximum entropy and outperform traditional product expansion methods in some cases.

Contribution

It introduces a variant belief network model that nearly guarantees maximum entropy and can achieve higher performance scores than standard product expansion.

Findings

A variant model exhibits near-maximum entropy guarantees.

The variant model outperforms product expansion in many cases.

The product expansion of conditional probabilities is not maximum entropy.

Abstract

The product expansion of conditional probabilities for belief nets is not maximum entropy. This appears to deny a desirable kind of assurance for the model. However, a kind of guarantee that is almost as strong as maximum entropy can be derived. Surprisingly, a variant model also exhibits the guarantee, and for many cases obtains a higher performance score than the product expansion.