A Beta Cauchy-Cauchy (BECCA) shrinkage prior for Bayesian variable selection

TL;DR

This paper proposes a new Bayesian variable selection method using a Beta Cauchy-Cauchy (BECCA) prior, which improves predictive performance and allows efficient inference in high-dimensional regression models.

Contribution

It introduces a continuous Beta-distributed prior replacing indicator variables, combined with half Cauchy priors, enabling fast gradient-based inference and explicit variable selection.

Findings

Outperforms existing Bayesian methods in simulations.

Effective in linear and logistic regression.

Validated on real datasets.

Abstract

This paper introduces a novel Bayesian approach for variable selection in high-dimensional and potentially sparse regression settings. Our method replaces the indicator variables in the traditional spike and slab prior with continuous, Beta-distributed random variables and places half Cauchy priors over the parameters of the Beta distribution, which significantly improves the predictive and inferential performance of the technique. Similar to shrinkage methods, our continuous parameterization of the spike and slab prior enables us explore the posterior distributions of interest using fast gradient-based methods, such as Hamiltonian Monte Carlo (HMC), while at the same time explicitly allowing for variable selection in a principled framework. We study the frequentist properties of our model via simulation and show that our technique outperforms the latest Bayesian variable selection methods in both linear and logistic regression. The efficacy, applicability and performance of our approach, are further underscored through its implementation on real datasets.