A New Bayesian Approach to Robustness Against Outliers in Linear Regression

TL;DR

This paper introduces a Bayesian linear regression model with super heavy-tailed errors that automatically diminishes the influence of outliers, providing robust and efficient estimates with practical implementation guidance.

Contribution

It proposes a novel super heavy-tailed error model for Bayesian linear regression that ensures robustness against outliers, improving upon traditional methods.

Findings

Model is fully robust to outliers as they move further away

Estimation procedure remains efficient and easy to compute

Simulation demonstrates superior performance over standard approaches

Abstract

Linear regression is ubiquitous in statistical analysis. It is well understood that conflicting sources of information may contaminate the inference when the classical normality of errors is assumed. The contamination caused by the light normal tails follows from an undesirable effect: the posterior concentrates in an area in between the different sources with a large enough scaling to incorporate them all. The theory of conflict resolution in Bayesian statistics (O'Hagan and Pericchi (2012)) recommends to address this problem by limiting the impact of outliers to obtain conclusions consistent with the bulk of the data. In this paper, we propose a model with super heavy-tailed errors to achieve this. We prove that it is wholly robust, meaning that the impact of outliers gradually vanishes as they move further and further away form the general trend. The super heavy-tailed density is similar to the normal outside of the tails, which gives rise to an efficient estimation procedure. In addition, estimates are easily computed. This is highlighted via a detailed user guide, where all steps are explained through a simulated case study. The performance is shown using simulation. All required code is given.