Improving the precision of classification trees

TL;DR

This paper proposes four techniques to enhance the accuracy of classification trees, addressing issues like variable selection bias and local search limitations, and compares their effectiveness with other algorithms on various datasets.

Contribution

It introduces four novel methods to improve classification tree precision, overcoming biases and local search issues, and evaluates their performance against existing algorithms.

Findings

Improved variable selection accuracy

Enhanced model interpretability

Competitive performance with ensemble methods

Abstract

Besides serving as prediction models, classification trees are useful for finding important predictor variables and identifying interesting subgroups in the data. These functions can be compromised by weak split selection algorithms that have variable selection biases or that fail to search beyond local main effects at each node of the tree. The resulting models may include many irrelevant variables or select too few of the important ones. Either eventuality can lead to erroneous conclusions. Four techniques to improve the precision of the models are proposed and their effectiveness compared with that of other algorithms, including tree ensembles, on real and simulated data sets.