Why Is Diagnosis Using Belief Networks Insensitive to Imprecision In Probabilities?

TL;DR

This paper investigates why diagnostic accuracy using Bayesian belief networks remains stable despite significant imprecision in probability estimates, highlighting the robustness of such networks in medical diagnosis.

Contribution

The study provides empirical evidence and analysis explaining the insensitivity of belief network diagnostics to probability imprecision, including effects of symmetric and asymmetric noise.

Findings

Diagnostic performance shows small sensitivity to large probability noise.

Gold-standard posterior probabilities near zero or one are minimally affected by noise.

Symmetric and asymmetric noise have modest effects on diagnostic accuracy.

Abstract

Recent research has found that diagnostic performance with Bayesian belief networks is often surprisingly insensitive to imprecision in the numerical probabilities. For example, the authors have recently completed an extensive study in which they applied random noise to the numerical probabilities in a set of belief networks for medical diagnosis, subsets of the CPCS network, a subset of the QMR (Quick Medical Reference) focused on liver and bile diseases. The diagnostic performance in terms of the average probabilities assigned to the actual diseases showed small sensitivity even to large amounts of noise. In this paper, we summarize the findings of this study and discuss possible explanations of this low sensitivity. One reason is that the criterion for performance is average probability of the true hypotheses, rather than average error in probability, which is insensitive to symmetric noise distributions. But, we show that even asymmetric, logodds-normal noise has modest effects. A second reason is that the gold-standard posterior probabilities are often near zero or one, and are little disturbed by noise.