The Lasso Distribution: Properties, Sampling Methods, and Applications in Bayesian Lasso Regression

TL;DR

This paper introduces the Lasso distribution, explores its properties, and demonstrates its application in Bayesian Lasso regression, leading to more efficient sampling methods and deeper understanding of the Lasso penalty's probabilistic structure.

Contribution

We define the Lasso distribution, derive its properties, develop a sampling algorithm, and integrate it into Bayesian regression models, enhancing computational efficiency and theoretical understanding.

Findings

Derived closed-form moments and MGF of the Lasso distribution

Developed a stable sampling algorithm for the distribution

Improved Bayesian Lasso inference with the new distribution

Abstract

In this paper, we introduce a new probability distribution, the Lasso distribution. We derive several fundamental properties of the distribution, including closed-form expressions for its moments and moment-generating function. Additionally, we present an efficient and numerically stable algorithm for generating random samples from the distribution, facilitating its use in both theoretical and applied settings. We establish that the Lasso distribution belongs to the exponential family. A direct application of the Lasso distribution arises in the context of an existing Gibbs sampler, where the full conditional distribution of each regression coefficient follows this distribution. This leads to a more computationally efficient and theoretically grounded sampling scheme. To facilitate the adoption of our methodology, we provide an R package, BayesianLasso, available on CRAN, implementing the proposed methods. Our findings offer new insights into the probabilistic structure underlying the Lasso penalty and provide practical improvements in Bayesian inference for high-dimensional regression problems.