Fully Bayesian Penalized Regression with a Generalized Bridge Prior

TL;DR

This paper introduces a fully Bayesian penalized regression framework using a generalized bridge prior, enabling robust variable selection and prediction across sparse and dense models, with proven consistency and an efficient sampling algorithm.

Contribution

It develops a unified Bayesian approach for penalized regression that automatically eliminates nuisance parameters and adapts to different sparsity levels, improving inference and model selection.

Findings

Method achieves tail robustness and posterior consistency.

Performs comparably or better than existing methods in simulations.

Effectively selects optimal penalties in real data applications.

Abstract

We consider penalized regression models under a unified framework where the particular method is determined by the form of the penalty term. We propose a fully Bayesian approach that incorporates both sparse and dense settings and show how to use a type of model averaging approach to eliminate the nuisance penalty parameters and perform inference through the marginal posterior distribution of the regression coefficients. We establish tail robustness of the resulting estimator as well as conditional and marginal posterior consistency. We develop an efficient component-wise Markov chain Monte Carlo algorithm for sampling. Numerical results show that the method tends to select the optimal penalty and performs well in both variable selection and prediction and is comparable to, and often better than alternative methods. Both simulated and real data examples are provided.