Robust Bayesian Nonparametric Variable Selection for Linear Regression

TL;DR

This paper introduces a Bayesian nonparametric method for linear regression that robustly performs variable selection while handling outliers and heteroskedasticity, improving over traditional methods.

Contribution

It develops a Dirichlet process scale mixture model with closed-form conditionals, enabling efficient inference and extension to heavy-tailed responses.

Findings

Outperforms existing methods on synthetic datasets

Effective in real-world applications with outliers

Handles heteroskedasticity and heavy tails

Abstract

Spike-and-slab and horseshoe regression are arguably the most popular Bayesian variable selection approaches for linear regression models. However, their performance can deteriorate if outliers and heteroskedasticity are present in the data, which are common features in many real-world statistics and machine learning applications. In this work, we propose a Bayesian nonparametric approach to linear regression that performs variable selection while accounting for outliers and heteroskedasticity. Our proposed model is an instance of a Dirichlet process scale mixture model with the advantage that we can derive the full conditional distributions of all parameters in closed form, hence producing an efficient Gibbs sampler for posterior inference. Moreover, we present how to extend the model to account for heavy-tailed response variables. The performance of the model is tested against competing algorithms on synthetic and real-world datasets.