Neuronized Priors for Bayesian Sparse Linear Regression

TL;DR

This paper introduces neuronized priors for Bayesian sparse linear regression, unifying and extending popular shrinkage priors, improving computational efficiency, and providing theoretical guarantees for posterior contraction and convergence.

Contribution

It proposes a novel neuronized prior framework that simplifies variable selection and enhances computational efficiency in Bayesian linear regression.

Findings

Neuronized priors achieve the same variable selection as spike-and-slab without latent indicators.

Theoretical conditions ensure optimal posterior contraction rates.

MCMC algorithms under neuronized priors converge exponentially fast.

Abstract

Although Bayesian variable selection methods have been intensively studied, their routine use in practice has not caught up with their non-Bayesian counterparts such as Lasso, likely due to difficulties in both computations and flexibilities of prior choices. To ease these challenges, we propose the neuronized priors to unify and extend some popular shrinkage priors, such as Laplace, Cauchy, horseshoe, and spike-and-slab priors. A neuronized prior can be written as the product of a Gaussian weight variable and a scale variable transformed from Gaussian via an activation function. Compared with classic spike-and-slab priors, the neuronized priors achieve the same explicit variable selection without employing any latent indicator variables, which results in both more efficient and flexible posterior sampling and more effective posterior modal estimation. Theoretically, we provide specific conditions on the neuronized formulation to achieve the optimal posterior contraction rate, and show that a broadly applicable MCMC algorithm achieves an exponentially fast convergence rate under the neuronized formulation. We also examine various simulated and real data examples and demonstrate that using the neuronization representation is computationally more or comparably efficient than its standard counterpart in all well-known cases. An R package NPrior is provided in the CRAN for using neuronized priors in Bayesian linear regression.