Uncertainty quantification for the horseshoe

TL;DR

This paper analyzes the uncertainty quantification properties of the horseshoe prior in sparse normal means models, demonstrating conditions under which credible sets have valid frequentist coverage and discussing limitations due to over-shrinkage.

Contribution

It provides a theoretical investigation of the frequentist coverage of Bayesian credible sets under the horseshoe prior without known sparsity, highlighting conditions for validity and limitations.

Findings

Credible sets have good coverage if sparsity is correctly specified.

Hierarchical and empirical Bayes credible sets are not honest over the full parameter space.

Proper cutoff choices control false discoveries.

Abstract

We investigate the credible sets and marginal credible intervals resulting from the horseshoe prior in the sparse multivariate normal means model. We do so in an adaptive setting without assuming knowledge of the sparsity level (number of signals). We consider both the hierarchical Bayes method of putting a prior on the unknown sparsity level and the empirical Bayes method with the sparsity level estimated by maximum marginal likelihood. We show that credible balls and marginal credible intervals have good frequentist coverage and optimal size if the sparsity level of the prior is set correctly. By general theory honest confidence sets cannot adapt in size to an unknown sparsity level. Accordingly the hierarchical and empirical Bayes credible sets based on the horseshoe prior are not honest over the full parameter space. We show that this is due to over-shrinkage for certain parameters and characterise the set of parameters for which credible balls and marginal credible intervals do give correct uncertainty quantification. In particular we show that the fraction of false discoveries by the marginal Bayesian procedure is controlled by a correct choice of cut-off.