Experimental Comparison of Classification Uncertainty for Randomised and Bayesian Decision Tree Ensembles

TL;DR

This paper experimentally compares the classification uncertainty of randomised and Bayesian decision tree ensembles, demonstrating that Bayesian methods provide superior uncertainty estimates on synthetic and real datasets.

Contribution

It introduces an evaluation method for classification uncertainty and shows Bayesian decision trees outperform randomised ensembles in uncertainty estimation.

Findings

Bayesian decision trees have better uncertainty estimation.

The Uncertainty Envelope technique effectively evaluates classifier confidence.

Bayesian methods outperform randomised ensembles in uncertainty assessment.

Abstract

In this paper we experimentally compare the classification uncertainty of the randomised Decision Tree (DT) ensemble technique and the Bayesian DT technique with a restarting strategy on a synthetic dataset as well as on some datasets commonly used in the machine learning community. For quantitative evaluation of classification uncertainty, we use an Uncertainty Envelope dealing with the class posterior distribution and a given confidence probability. Counting the classifier outcomes, this technique produces feasible evaluations of the classification uncertainty. Using this technique in our experiments, we found that the Bayesian DT technique is superior to the randomised DT ensemble technique.