Empirical Bayes analysis of spike and slab posterior distributions

TL;DR

This paper studies the convergence properties of empirical Bayes methods using spike and slab priors in sparse normal means models, highlighting optimal choices and limitations of common priors.

Contribution

It demonstrates the convergence rates of empirical Bayes posteriors with various spike and slab priors, revealing the suboptimality of the Laplace slab in this context.

Findings

Convergence at minimax rate for certain spike and slab choices.

Heavy tailed slabs improve posterior convergence.

Laplace slab leads to suboptimal convergence rate.

Abstract

In the sparse normal means model, convergence of the Bayesian posterior distribution associated to spike and slab prior distributions is considered. The key sparsity hyperparameter is calibrated via marginal maximum likelihood empirical Bayes. The plug-in posterior squared-$L^2$ norm is shown to converge at the minimax rate for the euclidean norm for appropriate choices of spike and slab distributions. Possible choices include standard spike and slab with heavy tailed slab, and the spike and slab LASSO of Rockov\'a and George with heavy tailed slab. Surprisingly, the popular Laplace slab is shown to lead to a suboptimal rate for the full empirical Bayes posterior. This provides a striking example where convergence of aspects of the empirical Bayes posterior does not entail convergence of the full empirical Bayes posterior itself.