MOB-ESP and other Improvements in Probability Estimation

TL;DR

This paper introduces MOB-ESP, an ensemble algorithm that significantly improves class probability estimates over existing methods, demonstrated through extensive benchmark comparisons and metrics related to prediction accuracy.

Contribution

The paper presents MOB-ESP, a novel ensemble-based probability estimation method that outperforms previous algorithms like BPETs and EB-PETs in accuracy and ranking.

Findings

MOB-ESP produces more accurate class probabilities than BPETs and EB-PETs.

MOB-ESP achieves better probability rankings.

Experimental results are validated on multiple benchmark datasets.

Abstract

A key prerequisite to optimal reasoning under uncertainty in intelligent systems is to start with good class probability estimates. This paper improves on the current best probability estimation trees (Bagged-PETs) and also presents a new ensemble-based algorithm (MOB-ESP). Comparisons are made using several benchmark datasets and multiple metrics. These experiments show that MOB-ESP outputs significantly more accurate class probabilities than either the baseline BPETs algorithm or the enhanced version presented here (EB-PETs). These results are based on metrics closely associated with the average accuracy of the predictions. MOB-ESP also provides much better probability rankings than B-PETs. The paper further suggests how these estimation techniques can be applied in concert with a broader category of classifiers.