Objective Model Prior Probabilities in Variable Selection

TL;DR

This paper examines the limitations of equal and Jeffreys' model priors in Bayesian variable selection, proposing and analyzing objective alternative priors to improve model uncertainty analysis.

Contribution

It introduces and evaluates new objective prior choices for Bayesian variable selection, addressing issues with traditional priors in large model spaces.

Findings

Equal priors lead to overly large models in big spaces

Jeffreys' prior has serious problems in practice

Proposes and analyzes alternative objective priors

Abstract

For many years it was routine to use equal model prior probabilities in Bayesian model uncertainty analysis. At least twenty years ago it became clear that this was problematic, leading to support of much too large models in the increasingly huge model spaces being considered in genomics and other fields. A popular replacement was to adopt a suggestion of Harold Jeffreys for the variable selection problem in which a total of $k$ possible variables are being considered for inclusion in the model: give the collection of all models containing $d$ variables ($d = 0, . . . , k$) prior probability $1/(k + 1)$ and then divide this prior probability equally among the models in the collection. Many other choices of model prior probabilities that impose severe parsimony have also been introduced. We begin by reviewing the problems with using equal model prior probabilities and then discuss some serious problems with the Jeffreys choice. Finally, we introduce and study a number of objective alternative choices of model prior probabilities, from both numerical and theoretical perspectives.