A robust Bayesian analysis of variable selection under prior ignorance

TL;DR

This paper introduces a robust Bayesian variable selection method that assesses sensitivity to prior assumptions using latent variables and sets of priors, improving high-dimensional model selection reliability.

Contribution

It develops a cautious Bayesian approach with sensitivity analysis for variable selection, incorporating sets of priors to evaluate prior influence on model outcomes.

Findings

Sensitivity analysis reveals the impact of prior choices on variable selection.

The method provides monotone posterior odds with respect to prior expectations.

Application to datasets demonstrates improved robustness over traditional methods.

Abstract

We propose a cautious Bayesian variable selection routine by investigating the sensitivity of a hierarchical model, where the regression coefficients are specified by spike and slab priors. We exploit the use of latent variables to understand the importance of the co-variates. These latent variables also allow us to obtain the size of the model space which is an important aspect of high dimensional problems. In our approach, instead of fixing a single prior, we adopt a specific type of robust Bayesian analysis, where we consider a set of priors within the same parametric family to specify the selection probabilities of these latent variables. We achieve that by considering a set of expected prior selection probabilities, which allows us to perform a sensitivity analysis to understand the effect of prior elicitation on the variable selection. The sensitivity analysis provides us sets of posteriors for the regression coefficients as well as the selection indicators and we show that the posterior odds of the model selection probabilities are monotone with respect to the prior expectations of the selection probabilities. We also analyse synthetic and real life datasets to illustrate our cautious variable selection method and compare it with other well known methods.