Random Planted Forest: a directly interpretable tree ensemble

TL;DR

The paper introduces a new interpretable tree ensemble method called random planted forest, which modifies traditional random forests to produce non-binary, interpretable trees with bounded interactions, showing promising prediction and visualization results.

Contribution

It proposes a novel tree ensemble algorithm that enhances interpretability by keeping leaves after splits and bounding interactions, with theoretical analysis of its convergence properties.

Findings

Encouraging prediction and visualization performance in simulations.

Theoretical results show asymptotic optimal convergence rates for low interaction bounds.

The method offers a flexible trade-off between interpretability and modeling complexity.

Abstract

We introduce a novel interpretable tree based algorithm for prediction in a regression setting. Our motivation is to estimate the unknown regression function from a functional decomposition perspective in which the functional components correspond to lower order interaction terms. The idea is to modify the random forest algorithm by keeping certain leaves after they are split instead of deleting them. This leads to non-binary trees which we refer to as planted trees. An extension to a forest leads to our random planted forest algorithm. Additionally, the maximum number of covariates which can interact within a leaf can be bounded. If we set this interaction bound to one, the resulting estimator is a sum of one-dimensional functions. In the other extreme case, if we do not set a limit, the resulting estimator and corresponding model place no restrictions on the form of the regression function. In a simulation study we find encouraging prediction and visualisation properties of our random planted forest method. We also develop theory for an idealized version of random planted forests in cases where the interaction bound is low. We show that if it is smaller than three, the idealized version achieves asymptotically optimal convergence rates up to a logarithmic factor. Code is available on GitHub https://github.com/PlantedML/randomPlantedForest.