Grafting: Making Random Forests Consistent

TL;DR

This paper investigates the theoretical consistency of Random Forests, proposing a grafting method onto shallow CART trees that guarantees consistency and demonstrates strong empirical performance.

Contribution

It introduces a grafting approach to make Random Forests consistent, providing theoretical guarantees and empirical validation.

Findings

Grafting onto shallow CART ensures consistency.

The method performs well empirically.

Provides theoretical guarantees for Random Forests.

Abstract

Despite their performance and widespread use, little is known about the theory of Random Forests. A major unanswered question is whether, or when, the Random Forest algorithm is consistent. The literature explores various variants of the classic Random Forest algorithm to address this question and known short-comings of the method. This paper is a contribution to this literature. Specifically, the suitability of grafting consistent estimators onto a shallow CART is explored. It is shown that this approach has a consistency guarantee and performs well in empirical settings.