Risk Bounds for CART Classifiers under a Margin Condition

TL;DR

This paper derives risk bounds for CART classifiers under a margin condition, validating the pruning and selection procedures, and demonstrating their effectiveness in binary classification tasks.

Contribution

It provides new risk bounds for CART classifiers under a margin condition, supporting the validity of the pruning algorithm and test-based selection.

Findings

Risk bounds are established under a margin condition.

The linear penalty in CART pruning is validated.

Test sample selection does not significantly affect accuracy.

Abstract

Risk bounds for Classification and Regression Trees (CART, Breiman et. al. 1984) classifiers are obtained under a margin condition in the binary supervised classification framework. These risk bounds are obtained conditionally on the construction of the maximal deep binary tree and permit to prove that the linear penalty used in the CART pruning algorithm is valid under a margin condition. It is also shown that, conditionally on the construction of the maximal tree, the final selection by test sample does not alter dramatically the estimation accuracy of the Bayes classifier. In the two-class classification framework, the risk bounds that are proved, obtained by using penalized model selection, validate the CART algorithm which is used in many data mining applications such as Biology, Medicine or Image Coding.