Bayesian Elastic Net Regression with Structured Prior Dependence

TL;DR

This paper introduces a structured prior dependence in Bayesian elastic net regression using the orthant normal distribution, enhancing modeling flexibility and demonstrating improved results through simulations and spectroscopy data.

Contribution

It proposes a novel structured prior for Bayesian elastic net regression and develops a Gibbs sampling method to efficiently perform inference.

Findings

Structured prior dependence improves model flexibility.

Gibbs sampling enables efficient posterior computation.

Enhanced performance demonstrated on real and simulated data.

Abstract

Many regularization priors for Bayesian regression assume the regression coefficients are a priori independent. In particular this is the case for standard Bayesian treatments of the lasso and the elastic net. While independence may be reasonable in some data-analytic settings, incorporating dependence in these prior distributions provides greater modeling flexibility. This paper introduces the orthant normal distribution in its general form and shows how it can be used to structure prior dependence in the Bayesian elastic net regression model. An L1-regularized version of Zellner's g prior is introduced as a special case, creating a new link between the literature on penalized optimization and an important class of regression priors. Computation is challenging due to an intractable normalizing constant in the prior. We avoid this issue by modifying slightly a standard prior of convenience for the hyperparameters in such a way to enable simple and fast Gibbs sampling of the posterior distribution. The benefit of including structured prior dependence in the Bayesian elastic net regression model is demonstrated through simulation and a near-infrared spectroscopy data example.