Heavy-tailed and Horseshoe priors for regression and sparse Besov rates

TL;DR

This paper introduces heavy-tailed and horseshoe priors for nonparametric regression, demonstrating their ability to adaptively achieve near-optimal posterior contraction rates over various function classes like Besov and Sobolev spaces.

Contribution

It extends the framework of heavy-tailed priors to nonparametric Bayesian regression, including the first analysis of horseshoe priors in this context, and derives adaptive minimax contraction rates.

Findings

Posterior contraction rates are established for Sobolev and Besov spaces.

The OT prior form is shown to be necessary for full adaptation.

Simulation results support theoretical findings.

Abstract

The large variety of functions encountered in nonparametric statistics, calls for methods that are flexible enough to achieve optimal or near-optimal performance over a wide variety of functional classes, such as Besov balls, as well as over a large array of loss functions. In this work, we show that a class of heavy-tailed prior distributions on basis function coefficients introduced in \cite{AC} and called Oversmoothed heavy-Tailed (OT) priors, leads to Bayesian posterior distributions that satisfy these requirements; the case of horseshoe distributions is also investigated, for the first time in the context of nonparametrics, and we show that they fit into this framework. Posterior contraction rates are derived in two settings. The case of Sobolev--smooth signals and $L_2$--risk is considered first, along with a lower bound result showing that the imposed form of the scalings on prior coefficients by the OT prior is necessary to get full adaptation to smoothness. Second, the broader case of Besov-smooth signals with $L_{p'}$--risks, $p' \geq 1$, is considered, and minimax posterior contraction rates, adaptive to the underlying smoothness, and including rates in the so-called {\em sparse} zone, are derived. We provide an implementation of the proposed method and illustrate our results through a simulation study.