Functional Horseshoe Priors for Subspace Shrinkage

TL;DR

The paper introduces the functional horseshoe prior (fHS), a new shrinkage prior for functions that promotes shrinkage towards parametric classes, improving model selection and estimation accuracy in nonparametric additive models.

Contribution

The paper proposes the functional horseshoe prior (fHS), a novel approach that shrinks functions towards parametric classes, with theoretical guarantees and superior empirical performance.

Findings

fHS achieves adaptive posterior concentration on functions

Model selection consistency is established for the thresholded fHS

fHS outperforms standard horseshoe and penalized likelihood methods in simulations and real data

Abstract

We introduce a new shrinkage prior on function spaces, called the functional horseshoe prior (fHS), that encourages shrinkage towards parametric classes of functions. Unlike other shrinkage priors for parametric models, the fHS shrinkage acts on the shape of the function rather than inducing sparsity on model parameters. We study the efficacy of the proposed approach by showing an adaptive posterior concentration property on the function. We also demonstrate consistency of the model selection procedure that thresholds the shrinkage parameter of the functional horseshoe prior. We apply the fHS prior to nonparametric additive models and compare its performance with procedures based on the standard horseshoe prior and several penalized likelihood approaches. We find that the new procedure achieves smaller estimation error and more accurate model selection than other procedures in several simulated and real examples. The supplementary material for this article, which contains additional simulated and real data examples, MCMC diagnostics, and proofs of the theoretical results, is available online.