Log-Linear Bayesian Additive Regression Trees for Multinomial Logistic and Count Regression Models

TL;DR

This paper extends Bayesian additive regression trees (BART) to handle log-linear models like multinomial logistic and count regression with zero-inflation, introducing new data augmentation and priors for efficient MCMC sampling.

Contribution

It develops novel data augmentation strategies and prior distributions enabling BART to be applied to non-Gaussian log-linear models, broadening its applicability.

Findings

Effective MCMC algorithms for non-Gaussian models

Successful application to real datasets

Enhanced flexibility in modeling count and multinomial data

Abstract

We introduce Bayesian additive regression trees (BART) for log-linear models including multinomial logistic regression and count regression with zero-inflation and overdispersion. BART has been applied to nonparametric mean regression and binary classification problems in a range of settings. However, existing applications of BART have been limited to models for Gaussian "data", either observed or latent. This is primarily because efficient MCMC algorithms are available for Gaussian likelihoods. But while many useful models are naturally cast in terms of latent Gaussian variables, many others are not -- including models considered in this paper. We develop new data augmentation strategies and carefully specified prior distributions for these new models. Like the original BART prior, the new prior distributions are carefully constructed and calibrated to be flexible while guarding against overfitting. Together the new priors and data augmentation schemes allow us to implement an efficient MCMC sampler outside the context of Gaussian models. The utility of these new methods is illustrated with examples and an application to a previously published dataset.