Inferences in Bayesian variable selection problems with large model spaces

TL;DR

This paper examines Bayesian variable selection in large model spaces, demonstrating that empirical frequency-based inferences via MCMC outperform search methods, with unbiased estimators providing reliable results.

Contribution

It shows that empirical frequency estimators in Bayesian variable selection are more effective than search-based methods, supported by theoretical explanation and empirical evidence.

Findings

Empirical frequency estimators are unbiased.

MCMC sampling outperforms search methods in large model spaces.

Two illustrative examples support the proposed approach.

Abstract

An important aspect of Bayesian model selection is how to deal with huge model spaces, since exhaustive enumeration of all the models entertained is unfeasible and inferences have to be based on the very small proportion of models visited. This is the case for the variable selection problem, with a moderate to large number of possible explanatory variables being considered in this paper. We review some of the strategies proposed in the literature and argue that inferences based on empirical frequencies via Markov Chain Monte Carlo sampling of the posterior distribution outperforms recently proposed searching methods. We give a plausible yet very simple explanation of this effect, showing that estimators based on frequencies are unbiased. The results obtained in two illustrative examples provide strong evidence in favor of our arguments.