Degrees of freedom in lasso problems

TL;DR

This paper derives the degrees of freedom for the lasso and generalized lasso fits without assumptions on the predictor matrix, extending previous results to more general settings and providing new insights into their properties.

Contribution

It extends the known degrees of freedom formula for the lasso to cases with arbitrary predictor matrices and generalized lasso, without requiring full rank assumptions.

Findings

Derived degrees of freedom for lasso with arbitrary predictor matrix

Extended degrees of freedom results to generalized lasso with penalty matrix D

Provided intermediate theoretical results on lasso and generalized lasso

Abstract

We derive the degrees of freedom of the lasso fit, placing no assumptions on the predictor matrix $X$. Like the well-known result of Zou, Hastie and Tibshirani [Ann. Statist. 35 (2007) 2173-2192], which gives the degrees of freedom of the lasso fit when $X$ has full column rank, we express our result in terms of the active set of a lasso solution. We extend this result to cover the degrees of freedom of the generalized lasso fit for an arbitrary predictor matrix $X$ (and an arbitrary penalty matrix $D$). Though our focus is degrees of freedom, we establish some intermediate results on the lasso and generalized lasso that may be interesting on their own.