Variable selection via penalized credible regions with Dirichlet-Laplace global-local shrinkage priors

TL;DR

This paper advances Bayesian variable selection by integrating Dirichlet-Laplace global-local shrinkage priors into the credible region framework, providing theoretical guarantees and a new hyperparameter tuning method based on R-square distribution matching.

Contribution

It introduces a novel combination of Dirichlet-Laplace priors with credible region selection, along with a practical hyperparameter tuning approach for high-dimensional linear regression.

Findings

Posterior consistency established for normal and DL priors.

Variable selection consistency demonstrated.

Hyperparameters tuned via R-square distribution matching.

Abstract

The method of Bayesian variable selection via penalized credible regions separates model fitting and variable selection. The idea is to search for the sparsest solution within the joint posterior credible regions. Although the approach was successful, it depended on the use of conjugate normal priors. More recently, improvements in the use of global-local shrinkage priors have been made for high-dimensional Bayesian variable selection. In this paper, we incorporate global-local priors into the credible region selection framework. The Dirichlet-Laplace (DL) prior is adapted to linear regression. Posterior consistency for the normal and DL priors are shown, along with variable selection consistency. We further introduce a new method to tune hyperparameters in prior distributions for linear regression. We propose to choose the hyperparameters to minimize a discrepancy between the induced distribution on R-square and a prespecified target distribution. Prior elicitation on R-square is more natural, particularly when there are a large number of predictor variables in which elicitation on that scale is not feasible. For a normal prior, these hyperparameters are available in closed form to minimize the Kullback-Leibler divergence between the distributions.