On the Posterior Computation Under the Dirichlet-Laplace Prior

TL;DR

This paper examines the implementation issues of Gibbs sampling algorithms for the Dirichlet-Laplace prior in high-dimensional Bayesian models, identifies inaccuracies, and proposes corrected, more efficient methods with theoretical guarantees.

Contribution

It uncovers critical implementation pitfalls in existing Gibbs sampling algorithms for the Dirichlet-Laplace prior and offers improved, theoretically sound sampling procedures.

Findings

Identified widespread inaccuracies in current Gibbs sampling implementations.

Demonstrated the practical impact of these inaccuracies on posterior inference.

Proposed corrected algorithms with better efficiency and theoretical guarantees.

Abstract

Modern applications routinely collect high-dimensional data, leading to statistical models having more parameters than there are samples available. A common solution is to impose sparsity in parameter estimation, often using penalized optimization methods. Bayesian approaches provide a probabilistic framework to formally quantify uncertainty through shrinkage priors. Among these, the Dirichlet-Laplace prior has attained recognition for its theoretical guarantees and wide applicability. This article identifies a critical yet overlooked issue in the implementation of Gibbs sampling algorithms for such priors. We demonstrate that ambiguities in the presentation of key algorithmic steps, while mathematically coherent, have led to widespread implementation inaccuracies that fail to target the intended posterior distribution -- a target endowed with rigorous asymptotic guarantees. Using the normal-means problem and high-dimensional linear regressions as canonical examples, we clarify these implementation pitfalls and their practical consequences and propose corrected and more efficient sampling procedures.