Experimentally Comparing Uncertain Inference Systems to Probability

TL;DR

This paper compares different uncertain inference systems, including Mycin and probability-based methods, highlighting their biases, performance variability, and the robustness of independence assumptions, advocating for Minimum Cross Entropy inference.

Contribution

It provides an axiomatic argument for using Minimum Cross Entropy inference and evaluates the performance and biases of Mycin, its variant, and simplified probability systems.

Findings

All systems generally gave accurate results.

Conditional independence assumptions yielded the most robust performance.

Biases can be quantitatively assessed and ranked for system selection.

Abstract

This paper examines the biases and performance of several uncertain inference systems: Mycin, a variant of Mycin. and a simplified version of probability using conditional independence assumptions. We present axiomatic arguments for using Minimum Cross Entropy inference as the best way to do uncertain inference. For Mycin and its variant we found special situations where its performance was very good, but also situations where performance was worse than random guessing, or where data was interpreted as having the opposite of its true import We have found that all three of these systems usually gave accurate results, and that the conditional independence assumptions gave the most robust results. We illustrate how the Importance of biases may be quantitatively assessed and ranked. Considerations of robustness might be a critical factor is selecting UlS's for a given application.