Consistency of Bayes factors for linear models

TL;DR

This paper investigates the conditions under which Bayes factors remain consistent in linear models as the number of regressors and sample size grow, highlighting the impact of prior choices on model selection reliability.

Contribution

It provides a comprehensive analysis of Bayes factor consistency in high-dimensional linear models, revealing how prior selection affects inconsistency.

Findings

Some priors cause inconsistency with finite regressors

Different priors lead to varying inconsistency levels as regressors grow

Posterior model probabilities help compare priors for variable selection

Abstract

The quality of a Bayes factor crucially depends on the number of regressors, the sample size and the prior on the regression parameters, and hence it has to be established in a case-by-case basis. In this paper we analyze the consistency of a wide class of Bayes factors when the number of potential regressors grows as the sample size grows. We have found that when the number of regressors is finite some classes of priors yield inconsistency, and\ when the potential number of regressors grows at the same rate than the sample size different priors yield different degree of inconsistency. For moderate sample sizes, we evaluate the Bayes factors by comparing the posterior model probability. This gives valuable information to discriminate between the priors for the model parameters commonly used for variable selection.