Horseshoe Priors and MDP

TL;DR

This paper connects foundational properties of the horseshoe prior with the asymptotic moderate deviation principle, revealing how its singularity influences sparsity and risk in Bayesian models.

Contribution

It demonstrates that the horseshoe prior's properties are finite-sample precursors to the asymptotic MDP, unifying prior behavior, super-efficiency, and risk under a logarithmic framework.

Findings

Horseshoe prior's singularity determines the MDP threshold.

Super-efficiency occurs below the threshold, tail robustness above it.

Bayes risk aligns with a logarithmic budget principle.

Abstract

Carvalho (2010) established two foundational theorems for the horseshoe prior: tight two-sided logarithmic bounds on the marginal density near the origin (Theorem~1.1), and a super-efficient rate of convergence of the Bayes predictive density to the true sampling density in sparse situations (Theorem~2). The ``Shrink Globally, Act Locally'' paper \citep{polson2010shrink} formalised necessary and sufficient conditions on the prior's behaviour at the origin for sparsity adaptation as $p \to \infty$. We show that these results are not merely descriptive properties of the horseshoe -- they are the finite-sample precursors to the asymptotic moderate deviation principle (MDP) of \citet{datta2026newlook}. The log-pole singularity $\piH(\theta) \asymp -\log\abs{\theta}$ is precisely the origin integrability boundary that selects the MDP threshold $\tcrit = \sqrt{\log(\pi n/2)}$; super-efficiency below the threshold and tail robustness above it together produce the ABOS Bayes risk $p_0 \log(p/p_0)/n$; and the Clarke--Barron information-theoretic asymptotics of Bayes methods provide the unifying framework in which all three results are faces of a single logarithmic budget principle.