On the Convergence of CART under Sufficient Impurity Decrease Condition

TL;DR

This paper analyzes the convergence rate of CART decision trees in regression, establishing an improved error bound under the sufficient impurity decrease condition and providing verifiable conditions for its satisfaction.

Contribution

It improves the theoretical understanding of CART's convergence under SID and introduces practical conditions for verifying SID in additive models.

Findings

Established an upper bound on CART prediction error under SID

Demonstrated the error bound cannot be significantly improved

Provided verifiable conditions for SID in additive models

Abstract

The decision tree is a flexible machine learning model that finds its success in numerous applications. It is usually fitted in a recursively greedy manner using CART. In this paper, we investigate the convergence rate of CART under a regression setting. First, we establish an upper bound on the prediction error of CART under a sufficient impurity decrease (SID) condition \cite{chi2022asymptotic} -- our result improves upon the known result by \cite{chi2022asymptotic} under a similar assumption. Furthermore, we provide examples that demonstrate the error bound cannot be further improved by more than a constant or a logarithmic factor. Second, we introduce a set of easily verifiable sufficient conditions for the SID condition. Specifically, we demonstrate that the SID condition can be satisfied in the case of an additive model, provided that the component functions adhere to a ``locally reverse Poincar{\'e} inequality". We discuss several well-known function classes in non-parametric estimation to illustrate the practical utility of this concept.