On Bayesian robust regression with diverging number of predictors

TL;DR

This paper introduces a Bayesian approach for robust regression with a diverging number of predictors, proposing a mixture prior and Gibbs sampling, which outperforms traditional methods in estimating the coefficient norm.

Contribution

It develops a Bayesian framework with a mixture prior and Gibbs sampler for high-dimensional robust regression, addressing limitations of M-estimators.

Findings

Bayesian estimator is consistent in Euclidean norm.

Simulation shows Bayesian method outperforms traditional estimators.

Proposed model effectively handles diverging predictors and observations.

Abstract

This paper concerns the robust regression model when the number of predictors and the number of observations grow in a similar rate. Theory for M-estimators in this regime has been recently developed by several authors [El Karoui et al., 2013, Bean et al., 2013, Donoho and Montanari, 2013]. Motivated by the inability of M-estimators to successfully estimate the Euclidean norm of the coefficient vector, we consider a Bayesian framework for this model. We suggest a two-component mixture of normals prior for the coefficients and develop a Gibbs sampler procedure for sampling from relevant posterior distributions, while utilizing a scale mixture of normal representation for the error distribution . Unlike M-estimators, the proposed Bayes estimator is consistent in the Euclidean norm sense. Simulation results demonstrate the superiority of the Bayes estimator over traditional estimation methods.