Variable Selection Using Bayesian Additive Regression Trees

TL;DR

This paper reviews variable selection methods for Bayesian additive regression trees (BART), introduces new importance measures and procedures, and demonstrates improved predictor recovery in simulations.

Contribution

It proposes novel permutation-based importance measures and a backward selection method tailored for BART, enhancing variable selection accuracy.

Findings

Improved predictor recovery in simulations

Effective handling of mixed predictor types

Enhanced variable selection performance

Abstract

Variable selection is an important statistical problem. This problem becomes more challenging when the candidate predictors are of mixed type (e.g. continuous and binary) and impact the response variable in nonlinear and/or non-additive ways. In this paper, we review existing variable selection approaches for the Bayesian additive regression trees (BART) model, a nonparametric regression model, which is flexible enough to capture the interactions between predictors and nonlinear relationships with the response. An emphasis of this review is on the capability of identifying relevant predictors. We also propose two variable importance measures which can be used in a permutation-based variable selection approach, and a backward variable selection procedure for BART. We present simulations demonstrating that our approaches exhibit improved performance in terms of the ability to recover all the relevant predictors in a variety of data settings, compared to existing BART-based variable selection methods.