Bayesian Distance Weighted Discrimination

TL;DR

This paper introduces a Bayesian framework for Distance Weighted Discrimination (DWD), enabling statistical inference, uncertainty quantification, and automatic tuning, with demonstrated benefits in high-dimensional classification tasks like genomics.

Contribution

It formulates DWD as a Bayesian model with a proper posterior, develops an MCMC algorithm for inference, and shows advantages over traditional DWD in uncertainty assessment and parameter tuning.

Findings

Bayesian DWD provides well-calibrated posterior class probabilities.

Uncertainty in coefficients and scores can be effectively quantified.

Semi-supervised analysis improves classification power.

Abstract

Distance weighted discrimination (DWD) is a linear discrimination method that is particularly well-suited for classification tasks with high-dimensional data. The DWD coefficients minimize an intuitive objective function, which can solved very efficiently using state-of-the-art optimization techniques. However, DWD has not yet been cast into a model-based framework for statistical inference. In this article we show that DWD identifies the mode of a proper Bayesian posterior distribution, that results from a particular link function for the class probabilities and a shrinkage-inducing proper prior distribution on the coefficients. We describe a relatively efficient Markov chain Monte Carlo (MCMC) algorithm to simulate from the true posterior under this Bayesian framework. We show that the posterior is asymptotically normal and derive the mean and covariance matrix of its limiting distribution. Through several simulation studies and an application to breast cancer genomics we demonstrate how the Bayesian approach to DWD can be used to (1) compute well-calibrated posterior class probabilities, (2) assess uncertainty in the DWD coefficients and resulting sample scores, (3) improve power via semi-supervised analysis when not all class labels are available, and (4) automatically determine a penalty tuning parameter within the model-based framework. R code to perform Bayesian DWD is available at https://github.com/lockEF/BayesianDWD .