Default Bayesian analysis with global-local shrinkage priors

TL;DR

This paper introduces a framework for evaluating the default suitability of global-local shrinkage priors, demonstrating that horseshoe priors are appropriate for non-informative Bayesian analysis due to their regular variation properties.

Contribution

It establishes a regular variation-based framework for assessing default priors and shows horseshoe priors are suitable for high-dimensional nonlinear parameter estimation.

Findings

Horseshoe priors possess regular variation, making them suitable for default Bayesian analysis.

Global-local shrinkage priors effectively separate signals from noise in high-dimensional models.

Horseshoe and horseshoe+ perform well in nonlinear high-dimensional problems as non-informative priors.

Abstract

We provide a framework for assessing the default nature of a prior distribution using the property of regular variation, which we study for global-local shrinkage priors. In particular, we demonstrate the horseshoe priors, originally designed to handle sparsity, also possess regular variation and thus are appropriate for default Bayesian analysis. To illustrate our methodology, we solve a problem of non-informative priors due to Efron (1973), who showed standard flat non-informative priors in high-dimensional normal means model can be highly informative for nonlinear parameters of interest. We consider four such problems and show global-local shrinkage priors such as the horseshoe and horseshoe+ perform as Efron (1973) requires in each case. We find the reason for this lies in the ability of the global-local shrinkage priors to separate a low-dimensional signal embedded in high-dimensional noise, even for nonlinear functions.