Hierarchical Embedded Bayesian Additive Regression Trees

TL;DR

HE-BART extends Bayesian Additive Regression Trees by incorporating random effects at the terminal node level, offering a flexible, non-parametric alternative to mixed effects models with improved prediction accuracy and reliable uncertainty estimates.

Contribution

The paper introduces HE-BART, a novel hierarchical extension of BART that models random effects non-parametrically without requiring predefined structure.

Findings

Superior prediction performance on standard mixed effects datasets

Consistent estimation of random effect variances

Maintains uncertainty calibration of standard BART

Abstract

We propose a simple yet powerful extension of Bayesian Additive Regression Trees which we name Hierarchical Embedded BART (HE-BART). The model allows for random effects to be included at the terminal node level of a set of regression trees, making HE-BART a non-parametric alternative to mixed effects models which avoids the need for the user to specify the structure of the random effects in the model, whilst maintaining the prediction and uncertainty calibration properties of standard BART. Using simulated and real-world examples, we demonstrate that this new extension yields superior predictions for many of the standard mixed effects models' example data sets, and yet still provides consistent estimates of the random effect variances. In a future version of this paper, we outline its use in larger, more advanced data sets and structures.