BFTS: Thompson Sampling with Bayesian Additive Regression Trees

TL;DR

This paper introduces BFTS, a novel contextual bandit algorithm that integrates Bayesian Additive Regression Trees for improved exploration, providing theoretical guarantees and demonstrating superior empirical performance in health intervention settings.

Contribution

The paper presents BFTS, the first to incorporate BART into Thompson Sampling for contextual bandits, with proven regret bounds and practical effectiveness in behavioral intervention data.

Findings

Achieves state-of-the-art regret on tabular benchmarks.

Provides near-nominal uncertainty calibration.

Improves engagement rates by over 30% in a health trial.

Abstract

Contextual bandits are a core technology for personalized mobile health interventions, where decision-making requires adapting to complex, non-linear user behaviors. While Thompson Sampling (TS) is a preferred strategy for these problems, its performance hinges on the quality of the underlying reward model. Standard linear models suffer from high bias, while neural network approaches are often brittle and difficult to tune in online settings. Conversely, tree ensembles dominate tabular data prediction but typically rely on heuristic uncertainty quantification, lacking a principled probabilistic basis for TS. We propose Bayesian Forest Thompson Sampling (BFTS), the first contextual bandit algorithm to integrate Bayesian Additive Regression Trees (BART), a fully probabilistic sum-of-trees model, directly into the exploration loop. We prove that BFTS is theoretically sound, deriving an information-theoretic Bayesian regret bound of $\tilde{O}(\sqrt{T})$. As a complementary result, we establish frequentist minimax optimality for a "feel-good" variant, confirming the structural suitability of BART priors for non-parametric bandits. Empirically, BFTS achieves state-of-the-art regret on tabular benchmarks with near-nominal uncertainty calibration. Furthermore, in an offline policy evaluation on the Drink Less micro-randomized trial, BFTS improves engagement rates by over 30% compared to the deployed policy, demonstrating its practical effectiveness for behavioral interventions.