Local Gaussian process extrapolation for BART models with applications to causal inference

TL;DR

This paper introduces a novel Gaussian process-based extrapolation method for BART models, improving prediction accuracy and interval coverage outside training data ranges, especially in causal inference applications.

Contribution

It proposes grafting Gaussian processes onto BART leaf nodes for better extrapolation, addressing a key limitation of standard BART in out-of-sample prediction.

Findings

Superior predictive performance in simulations

More accurate prediction intervals outside training data

Effective in causal inference scenarios with limited data regions

Abstract

Bayesian additive regression trees (BART) is a semi-parametric regression model offering state-of-the-art performance on out-of-sample prediction. Despite this success, standard implementations of BART typically provide inaccurate prediction and overly narrow prediction intervals at points outside the range of the training data. This paper proposes a novel extrapolation strategy that grafts Gaussian processes to the leaf nodes in BART for predicting points outside the range of the observed data. The new method is compared to standard BART implementations and recent frequentist resampling-based methods for predictive inference. We apply the new approach to a challenging problem from causal inference, wherein for some regions of predictor space, only treated or untreated units are observed (but not both). In simulation studies, the new approach boasts superior performance compared to popular alternatives, such as Jackknife+.