Sharded Bayesian Additive Regression Trees

TL;DR

This paper introduces the Sharded Bayesian Additive Regression Trees (SBT), a novel model that partitions data using a sharding tree and Bayesian methods, with theoretical and experimental validation of its optimality and complexity.

Contribution

The paper develops a new randomized SBT model with a sharding tree structure, combining data partitioning and Bayesian regression in a unified tree-based framework.

Findings

Derives theoretical optimal weights for posterior contraction.

Proves worst-case complexity of the SBT model.

Demonstrates effectiveness through experiments.

Abstract

In this paper we develop the randomized Sharded Bayesian Additive Regression Trees (SBT) model. We introduce a randomization auxiliary variable and a sharding tree to decide partitioning of data, and fit each partition component to a sub-model using Bayesian Additive Regression Tree (BART). By observing that the optimal design of a sharding tree can determine optimal sharding for sub-models on a product space, we introduce an intersection tree structure to completely specify both the sharding and modeling using only tree structures. In addition to experiments, we also derive the theoretical optimal weights for minimizing posterior contractions and prove the worst-case complexity of SBT.