Consistency of Bayesian Linear Model Selection With a Growing Number of Parameters

TL;DR

This paper investigates the asymptotic behavior of Bayesian linear model selection as the number of parameters grows with the sample size, establishing conditions for consistent true model selection.

Contribution

It provides new theoretical conditions under which Bayesian model selection consistently identifies the true model in high-dimensional linear settings.

Findings

Posterior probability of the true model dominates incorrect models with high probability.

Posterior probability of the true model converges to one under certain conditions.

Examples illustrating when dominance holds but convergence fails.

Abstract

Linear models with a growing number of parameters have been widely used in modern statistics. One important problem about this kind of model is the variable selection issue. Bayesian approaches, which provide a stochastic search of informative variables, have gained popularity. In this paper, we will study the asymptotic properties related to Bayesian model selection when the model dimension $p$ is growing with the sample size $n$. We consider $p\le n$ and provide sufficient conditions under which: (1) with large probability, the posterior probability of the true model (from which samples are drawn) uniformly dominates the posterior probability of any incorrect models; and (2) with large probability, the posterior probability of the true model converges to one. Both (1) and (2) guarantee that the true model will be selected under a Bayesian framework. We also demonstrate several situations when (1) holds but (2) fails, which illustrates the difference between these two properties. Simulated examples are provided to illustrate the main results.