Posterior Model Consistency in Variable Selection as the Model Dimension Grows

TL;DR

This paper investigates the conditions under which Bayesian posterior model probabilities remain consistent as the number of potential regressors increases with sample size, highlighting the impact of prior choices.

Contribution

It analyzes the consistency of posterior model probabilities in high-dimensional regression, revealing how prior selection influences model selection reliability.

Findings

Some common priors lead to inconsistency in posterior model probabilities.

Mixtures of priors can improve posterior model consistency.

Bayesian pairwise procedures' error rates inform prior selection.

Abstract

Most of the consistency analyses of Bayesian procedures for variable selection in regression refer to pairwise consistency, that is, consistency of Bayes factors. However, variable selection in regression is carried out in a given class of regression models where a natural variable selector is the posterior probability of the models. In this paper we analyze the consistency of the posterior model probabilities when the number of potential regressors grows as the sample size grows. The novelty in the posterior model consistency is that it depends not only on the priors for the model parameters through the Bayes factor, but also on the model priors, so that it is a useful tool for choosing priors for both models and model parameters. We have found that some classes of priors typically used in variable selection yield posterior model inconsistency, while mixtures of these priors improve this undesirable behavior. For moderate sample sizes, we evaluate Bayesian pairwise variable selection procedures by comparing their frequentist Type I and II error probabilities. This provides valuable information to discriminate between the priors for the model parameters commonly used for variable selection.