On the "degrees of freedom" of the lasso

TL;DR

This paper demonstrates that the number of nonzero coefficients in the lasso provides an unbiased and asymptotically consistent estimate of its degrees of freedom, enabling efficient model selection.

Contribution

It establishes that the nonzero coefficients count is an unbiased estimator of the lasso's degrees of freedom without predictor assumptions, facilitating principled model selection.

Findings

Number of nonzero coefficients is an unbiased estimate of degrees of freedom.

The estimator is asymptotically consistent.

Enables efficient model selection using criteria like AIC and BIC.

Abstract

We study the effective degrees of freedom of the lasso in the framework of Stein's unbiased risk estimation (SURE). We show that the number of nonzero coefficients is an unbiased estimate for the degrees of freedom of the lasso--a conclusion that requires no special assumption on the predictors. In addition, the unbiased estimator is shown to be asymptotically consistent. With these results on hand, various model selection criteria--$C_p$, AIC and BIC--are available, which, along with the LARS algorithm, provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least-squares fit.