Very fast Bayesian Additive Regression Trees on GPU

TL;DR

This paper introduces a GPU-accelerated implementation of Bayesian Additive Regression Trees (BART) that significantly reduces computation time, making it competitive with XGBoost for large datasets.

Contribution

The paper presents a GPU-enabled BART implementation that is up to 200 times faster than CPU versions, broadening BART's practical applicability.

Findings

GPU implementation achieves up to 200x speedup

BART becomes competitive with XGBoost in runtime

Implementation available in Python package bartz

Abstract

Bayesian Additive Regression Trees (BART) is a nonparametric Bayesian regression technique based on an ensemble of decision trees. It is part of the toolbox of many statisticians. The overall statistical quality of the regression is typically higher than other generic alternatives, and it requires less manual tuning, making it a good default choice. However, it is a niche method compared to its natural competitor XGBoost, due to the longer running time, making sample sizes above 10,000-100,000 a nuisance. I present a GPU-enabled implementation of BART, faster by up to 200x relative to a single CPU core, making BART competitive in running time with XGBoost. This implementation is available in the Python package bartz.