A Bayesian Methodology for Estimating Uncertainty of Decisions in Safety-Critical Systems

TL;DR

This paper introduces a Bayesian approach with a novel prior for decision trees, improving uncertainty estimation and predictive accuracy in safety-critical systems, especially when prior structure information is unavailable.

Contribution

It proposes a new prior for decision trees in Bayesian MCMC, enhancing decision uncertainty estimation without requiring prior structural information.

Findings

Outperforms existing Bayesian methods in predictive accuracy

Provides a new procedure for selecting a single decision tree

Effective in safety-critical system applications

Abstract

Uncertainty of decisions in safety-critical engineering applications can be estimated on the basis of the Bayesian Markov Chain Monte Carlo (MCMC) technique of averaging over decision models. The use of decision tree (DT) models assists experts to interpret causal relations and find factors of the uncertainty. Bayesian averaging also allows experts to estimate the uncertainty accurately when a priori information on the favored structure of DTs is available. Then an expert can select a single DT model, typically the Maximum a Posteriori model, for interpretation purposes. Unfortunately, a priori information on favored structure of DTs is not always available. For this reason, we suggest a new prior on DTs for the Bayesian MCMC technique. We also suggest a new procedure of selecting a single DT and describe an application scenario. In our experiments on the Short-Term Conflict Alert data our technique outperforms the existing Bayesian techniques in predictive accuracy of the selected single DTs.