MLCBART: Multilabel Classification with Bayesian Additive Regression Trees

TL;DR

This paper introduces MLCBART, a Bayesian additive regression trees framework for multilabel classification that models complex label relationships and provides uncertainty quantification, improving prediction accuracy over existing methods.

Contribution

MLCBART is a novel Bayesian nonparametric approach that explicitly models label correlations and uncertainty in multilabel classification tasks.

Findings

MLCBART outperforms other models in predictive accuracy.

The model effectively captures complex label relationships.

Uncertainty quantification enhances interpretability of predictions.

Abstract

Multilabel Classification (MLC) deals with the simultaneous classification of multiple binary labels. The task is challenging because, not only may there be arbitrarily different and complex relationships between predictor variables and each label, but associations among labels may exist even after accounting for effects of predictor variables. In this paper, we present a Bayesian additive regression tree (BART) framework to model the problem. BART is a nonparametric and flexible model structure capable of uncovering complex relationships within the data. Our adaptation, MLCBART, assumes that labels arise from thresholding an underlying numeric scale, where a multivariate normal model allows explicit estimation of the correlation structure among labels. This enables the discovery of complicated relationships in various forms and improves MLC predictive performance. Our Bayesian framework not only enables uncertainty quantification for each predicted label, but our MCMC draws produce an estimated conditional probability distribution of label combinations for any predictor values. Simulation experiments demonstrate the effectiveness of the proposed model by comparing its performance with a set of models, including the oracle model with the correct functional form. Results show that our model predicts vectors of labels more accurately than other contenders and its performance is close to the oracle model. An example highlights how the method's ability to produce measures of uncertainty on predictions provides nuanced understanding of classification results.