Diagnosis of Multiple Faults: A Sensitivity Analysis

TL;DR

This study compares three diagnostic inference models on real clinical cases, finding that the noisy OR model most closely aligns with expert diagnoses, highlighting the importance of model refinement for accurate medical diagnosis.

Contribution

The paper evaluates and compares the accuracy of three diagnostic models, introducing the noisy OR model as a promising approach for multiple fault diagnosis in medicine.

Findings

Noisy OR model aligns best with expert diagnoses

Multimembership Bayes overestimates disease probabilities

Simple Bayes underestimates disease probabilities

Abstract

We compare the diagnostic accuracy of three diagnostic inference models: the simple Bayes model, the multimembership Bayes model, which is isomorphic to the parallel combination function in the certainty-factor model, and a model that incorporates the noisy OR-gate interaction. The comparison is done on 20 clinicopathological conference (CPC) cases from the American Journal of Medicine-challenging cases describing actual patients often with multiple disorders. We find that the distributions produced by the noisy OR model agree most closely with the gold-standard diagnoses, although substantial differences exist between the distributions and the diagnoses. In addition, we find that the multimembership Bayes model tends to significantly overestimate the posterior probabilities of diseases, whereas the simple Bayes model tends to significantly underestimate the posterior probabilities. Our results suggest that additional work to refine the noisy OR model for internal medicine will be worthwhile.