Shrinkage with shrunken shoulders: Gibbs sampling shrinkage model posteriors with guaranteed convergence rates
Akihiko Nishimura, Marc A. Suchard

TL;DR
This paper introduces a regularization method for continuous shrinkage priors that maintains computational efficiency and guarantees convergence rates of Gibbs samplers, enhancing robustness in sparse Bayesian logistic regression.
Contribution
It proposes a novel regularization approach for shrinkage priors that preserves their structure and provides theoretical guarantees on Gibbs sampler convergence rates.
Findings
Regularization leads to geometric ergodicity for broad classes of priors.
Under certain conditions, the Gibbs sampler achieves uniform ergodicity.
The method improves robustness in sparse logistic regression models.
Abstract
Use of continuous shrinkage priors -- with a "spike" near zero and heavy-tails towards infinity -- is an increasingly popular approach to induce sparsity in parameter estimates. When the parameters are only weakly identified by the likelihood, however, the posterior may end up with tails as heavy as the prior, jeopardizing robustness of inference. A natural solution is to "shrink the shoulders" of a shrinkage prior by lightening up its tails beyond a reasonable parameter range, yielding a regularized version of the prior. We develop a regularization approach which, unlike previous proposals, preserves computationally attractive structures of original shrinkage priors. We study theoretical properties of the Gibbs sampler on resulting posterior distributions, with emphasis on convergence rates of the P{\'o}lya-Gamma Gibbs sampler for sparse logistic regression. Our analysis shows that the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMarkov Chains and Monte Carlo Methods · Statistical Methods and Inference · Bayesian Methods and Mixture Models
