A divergence formula for regularization methods with an L2 constraint

TL;DR

This paper introduces a divergence formula for regularization methods with an L2 constraint, aiding in unbiased degrees of freedom estimation for parameter selection across various models.

Contribution

It derives a novel divergence formula applicable to multiple regularization techniques, extending from smoothing splines to ridge regression and functional linear regression.

Findings

Provides an unbiased estimate of degrees of freedom

Applicable to smoothing splines, penalized splines, and ridge regression

Enhances regularization parameter selection methods

Abstract

We derive a divergence formula for a group of regularization methods with an L2 constraint. The formula is useful for regularization parameter selection, because it provides an unbiased estimate for the number of degrees of freedom. We begin with deriving the formula for smoothing splines and then extend it to other settings such as penalized splines, ridge regression, and functional linear regression.