A Sensitivity Analysis of Pathfinder: A Follow-up Study

TL;DR

This paper presents a follow-up sensitivity analysis of Pathfinder, showing that small parameter variations now have minimal impact on performance, thereby extending decision analytic sensitivity analysis to complex AI settings.

Contribution

It refines previous sensitivity analysis of Pathfinder by adjusting parameter variation ranges, demonstrating improved robustness and applicability to complex AI systems.

Findings

Small parameter variations now minimally affect Pathfinder's performance

Refined analysis offers more realistic insights into system robustness

Supports extension of decision analytic sensitivity analysis to complex AI

Abstract

At last year?s Uncertainty in AI Conference, we reported the results of a sensitivity analysis study of Pathfinder. Our findings were quite unexpected-slight variations to Pathfinder?s parameters appeared to lead to substantial degradations in system performance. A careful look at our first analysis, together with the valuable feedback provided by the participants of last year?s conference, led us to conduct a follow-up study. Our follow-up differs from our initial study in two ways: (i) the probabilities 0.0 and 1.0 remained unchanged, and (ii) the variations to the probabilities that are close to both ends (0.0 or 1.0) were less than the ones close to the middle (0.5). The results of the follow-up study look more reasonable-slight variations to Pathfinder?s parameters now have little effect on its performance. Taken together, these two sets of results suggest a viable extension of a common decision analytic sensitivity analysis to the larger, more complex settings generally encountered in artificial intelligence.