The LASSO Estimator: Distributional Properties

TL;DR

This paper derives finite-sample distribution properties of the LASSO estimator for linear models with Gaussian noise, providing new insights beyond asymptotic results and extending previous orthogonal model analyses.

Contribution

It generalizes finite-sample distribution analysis of LASSO to non-orthogonal models and derives explicit characteristic functions and approximate densities.

Findings

Derived the finite sample characteristic function of LASSO

Obtained approximate marginal densities of LASSO components

Extended distributional analysis beyond orthogonal models

Abstract

The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behavior. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator.