An Experimental Comparison of Numerical and Qualitative Probabilistic Reasoning

TL;DR

This paper compares qualitative and numerical probabilistic reasoning methods in diagnostic tasks, finding infinitesimal probabilities perform well with small priors but less so with higher priors, indicating practical value for certain applications.

Contribution

It provides an experimental evaluation of infinitesimal probability schemes against numerical ones in a diagnostic context, highlighting their effectiveness for small prior probabilities.

Findings

Infinitesimal probabilities perform comparably to numerical methods in fault diagnosis.

Performance declines when prior fault probabilities exceed 0.03.

Infinitesimal methods are potentially valuable for diagnosing rare faults.

Abstract

Qualitative and infinitesimal probability schemes are consistent with the axioms of probability theory, but avoid the need for precise numerical probabilities. Using qualitative probabilities could substantially reduce the effort for knowledge engineering and improve the robustness of results. We examine experimentally how well infinitesimal probabilities (the kappa-calculus of Goldszmidt and Pearl) perform a diagnostic task - troubleshooting a car that will not start by comparison with a conventional numerical belief network. We found the infinitesimal scheme to be as good as the numerical scheme in identifying the true fault. The performance of the infinitesimal scheme worsens significantly for prior fault probabilities greater than 0.03. These results suggest that infinitesimal probability methods may be of substantial practical value for machine diagnosis with small prior fault probabilities.