Bayesian Networks, Total Variation and Robustness

TL;DR

This paper advocates using total variation distance instead of Kullback-Leibler measures for assessing robustness in Bayesian Networks, providing more transparent bounds and integrating robustness analysis into the model-building process.

Contribution

It introduces a formal, practical approach for robustness analysis in Bayesian Networks using total variation, improving upon existing methods based on Kullback-Leibler divergence.

Findings

Total variation bounds are simpler and more transparent.

Robustness considerations can be integrated during model construction.

Demonstrated on two real-world Bayesian Networks.

Abstract

Now that Bayesian Networks (BNs) have become widely used, an appreciation is developing of just how critical an awareness of the sensitivity and robustness of certain target variables are to changes in the model. When time resources are limited, such issues impact directly on the chosen level of complexity of the BN as well as the quantity of missing probabilities we are able to elicit. Currently most such analyses are performed once the whole BN has been elicited and are based on Kullback-Leibler information measures. In this paper we argue that robustness methods based instead on the familiar total variation distance provide simple and more useful bounds on robustness to misspecification which are both formally justifiable and transparent. We demonstrate how such formal robustness considerations can be embedded within the process of building a BN. Here we focus on two particular choices a modeller needs to make: the choice of the parents of each node and the number of levels to choose for each variable within the system. Our analyses are illustrated throughout using two BNs drawn from the recent literature.