The Bayesian Bridge
Nicholas G. Polson, James G. Scott, Jesse Windle

TL;DR
The paper introduces the Bayesian bridge estimator for regularized regression and classification, developing two mixture representations that enable efficient MCMC sampling and outperform classical methods in estimation and prediction.
Contribution
It presents a novel mixture representation for the Bayesian bridge model, including a new one for orthogonal problems, and demonstrates its practical advantages and implementation in R.
Findings
Bayesian bridge outperforms classical methods in estimation and prediction.
New mixture representation improves efficiency for orthogonal problems.
MCMC methods show excellent mixing, especially for the global scale parameter.
Abstract
We propose the Bayesian bridge estimator for regularized regression and classification. Two key mixture representations for the Bayesian bridge model are developed: (1) a scale mixture of normals with respect to an alpha-stable random variable; and (2) a mixture of Bartlett--Fejer kernels (or triangle densities) with respect to a two-component mixture of gamma random variables. Both lead to MCMC methods for posterior simulation, and these methods turn out to have complementary domains of maximum efficiency. The first representation is a well known result due to West (1987), and is the better choice for collinear design matrices. The second representation is new, and is more efficient for orthogonal problems, largely because it avoids the need to deal with exponentially tilted stable random variables. It also provides insight into the multimodality of the joint posterior distribution, a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBayesian Methods and Mixture Models · Statistical Methods and Bayesian Inference · Statistical Methods and Inference
