Generalized double Pareto shrinkage

TL;DR

This paper introduces a generalized double Pareto prior for Bayesian linear models that combines sparsity and heavy-tailed behavior, enabling effective shrinkage and inference with simple computation.

Contribution

It proposes a new prior bridging Laplace and Normal-Jeffreys' priors, with a straightforward Gibbs sampling algorithm for Bayesian shrinkage estimation.

Findings

The prior exhibits desirable spike and tail properties.

The MAP estimator shows favorable sparsity and regularization characteristics.

Simulation and application demonstrate improved performance.

Abstract

We propose a generalized double Pareto prior for Bayesian shrinkage estimation and inferences in linear models. The prior can be obtained via a scale mixture of Laplace or normal distributions, forming a bridge between the Laplace and Normal-Jeffreys' priors. While it has a spike at zero like the Laplace density, it also has a Student's $t$-like tail behavior. Bayesian computation is straightforward via a simple Gibbs sampling algorithm. We investigate the properties of the maximum a posteriori estimator, as sparse estimation plays an important role in many problems, reveal connections with some well-established regularization procedures, and show some asymptotic results. The performance of the prior is tested through simulations and an application.