Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem

TL;DR

This paper examines how Bayesian variable-selection methods automatically correct for multiple testing, compares empirical-Bayes and fully Bayesian approaches, and reveals significant asymptotic differences between them through theoretical and empirical analysis.

Contribution

It clarifies the conditions under which Bayesian methods perform multiplicity correction and characterizes a surprising asymptotic discrepancy between empirical-Bayes and fully Bayesian approaches.

Findings

Bayesian priors can automatically correct for multiplicity in variable selection.

Empirical-Bayes and fully Bayesian methods can differ significantly asymptotically.

A theorem characterizes the asymptotic discrepancy between the two approaches.

Abstract

This paper studies the multiplicity-correction effect of standard Bayesian variable-selection priors in linear regression. Our first goal is to clarify when, and how, multiplicity correction happens automatically in Bayesian analysis, and to distinguish this correction from the Bayesian Ockham's-razor effect. Our second goal is to contrast empirical-Bayes and fully Bayesian approaches to variable selection through examples, theoretical results and simulations. Considerable differences between the two approaches are found. In particular, we prove a theorem that characterizes a surprising aymptotic discrepancy between fully Bayes and empirical Bayes. This discrepancy arises from a different source than the failure to account for hyperparameter uncertainty in the empirical-Bayes estimate. Indeed, even at the extreme, when the empirical-Bayes estimate converges asymptotically to the true variable-inclusion probability, the potential for a serious difference remains.