Bayesian Additive Regression Tree Copula Processes for Scalable Distributional Prediction

TL;DR

This paper introduces a scalable Bayesian additive regression tree copula process that constructs flexible distributional predictions, combining copula processes with BART models for improved accuracy and efficiency in high-dimensional data.

Contribution

It develops a novel copula process framework integrated with BART, enabling scalable, flexible distributional modeling with theoretical guarantees and practical efficiency.

Findings

Enhanced distributional prediction accuracy over baseline models

Maintains scalability and computational efficiency in high-dimensional settings

Proven posterior consistency and convergence properties

Abstract

We show how to construct the implied copula process of response values from a Bayesian additive regression tree (BART) model with prior on the leaf node variances. This copula process, defined on the covariate space, can be paired with any marginal distribution for the dependent variable to construct a flexible distributional BART model. Bayesian inference is performed via Markov chain Monte Carlo on an augmented posterior, where we show that key sampling steps can be realized as those of Chipman et al. (2010), preserving scalability and computational efficiency even though the copula process is high dimensional. The posterior predictive distribution from the copula process model is derived in closed form as the push-forward of the posterior predictive distribution of the underlying BART model with an optimal transport map. Under suitable conditions, we establish posterior consistency for the regression function and posterior means and prove convergence in distribution of the predictive process and conditional expectation. Simulation studies demonstrate improved accuracy of distributional predictions compared to the original BART model and leading benchmarks. Applications to five real datasets with 506 to 515,345 observations and 8 to 90 covariates further highlight the efficacy and scalability of our proposed BART copula process model.