GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes

TL;DR

GP-BART enhances traditional BART by integrating Gaussian process priors, enabling better modeling of smoothness and covariance structures, which improves predictive performance in regression tasks.

Contribution

This paper introduces GP-BART, a novel extension of BART that incorporates Gaussian process priors for improved modeling of smoothness and covariance in regression.

Findings

GP-BART outperforms traditional BART in simulated data scenarios.

GP-BART demonstrates superior predictive accuracy on real-world datasets.

The model effectively captures complex covariance structures.

Abstract

The Bayesian additive regression trees (BART) model is an ensemble method extensively and successfully used in regression tasks due to its consistently strong predictive performance and its ability to quantify uncertainty. BART combines "weak" tree models through a set of shrinkage priors, whereby each tree explains a small portion of the variability in the data. However, the lack of smoothness and the absence of an explicit covariance structure over the observations in standard BART can yield poor performance in cases where such assumptions would be necessary. The Gaussian processes Bayesian additive regression trees (GP-BART) model is an extension of BART which addresses this limitation by assuming Gaussian process (GP) priors for the predictions of each terminal node among all trees. The model's effectiveness is demonstrated through applications to simulated and real-world data, surpassing the performance of traditional modeling approaches in various scenarios.