XBART: Accelerated Bayesian Additive Regression Trees

TL;DR

This paper introduces XBART, a faster and more memory-efficient version of Bayesian Additive Regression Trees that maintains high predictive accuracy, outperforming random forests and gradient boosting in simulations.

Contribution

The paper develops a stochastic hill climbing algorithm for BART, significantly accelerating posterior estimation while preserving predictive performance.

Findings

Comparable computation time to existing methods

More accurate function estimation than random forests

Faster and less memory intensive than previous BART implementations

Abstract

Bayesian additive regression trees (BART) (Chipman et. al., 2010) is a powerful predictive model that often outperforms alternative models at out-of-sample prediction. BART is especially well-suited to settings with unstructured predictor variables and substantial sources of unmeasured variation as is typical in the social, behavioral and health sciences. This paper develops a modified version of BART that is amenable to fast posterior estimation. We present a stochastic hill climbing algorithm that matches the remarkable predictive accuracy of previous BART implementations, but is many times faster and less memory intensive. Simulation studies show that the new method is comparable in computation time and more accurate at function estimation than both random forests and gradient boosting.