Adaptive posterior contraction rates for the horseshoe

TL;DR

This paper demonstrates that the horseshoe prior, combined with empirical and hierarchical Bayes methods, achieves near minimax optimal estimation and rate-adaptive credible sets in sparse multivariate normal models.

Contribution

It introduces a data-driven approach to estimate sparsity, proving that the horseshoe prior adapts to unknown sparsity levels for optimal Bayesian inference.

Findings

MMLE effectively estimates sparsity level

Both Bayesian methods achieve rate-adaptive optimal contraction

Horseshoe posterior suitable for credible sets

Abstract

We investigate the frequentist properties of Bayesian procedures for estimation based on the horseshoe prior in the sparse multivariate normal means model. Previous theoretical results assumed that the sparsity level, that is, the number of signals, was known. We drop this assumption and characterize the behavior of the maximum marginal likelihood estimator (MMLE) of a key parameter of the horseshoe prior. We prove that the MMLE is an effective estimator of the sparsity level, in the sense that it leads to (near) minimax optimal estimation of the underlying mean vector generating the data. Besides this empirical Bayes procedure, we consider the hierarchical Bayes method of putting a prior on the unknown sparsity level as well. We show that both Bayesian techniques lead to rate-adaptive optimal posterior contraction, which implies that the horseshoe posterior is a good candidate for generating rate-adaptive credible sets.