Smoothly Adaptively Centered Ridge Estimator

TL;DR

This paper introduces SACR, a novel penalization framework for linear models with smooth covariates that adaptively reweight the ridge penalty to improve interpretability and prediction, demonstrated through simulations and spectroscopy applications.

Contribution

It proposes a convex, non-iterative method for jointly estimating coefficients and weight functions, enabling adaptive, smooth, and sparse solutions in ridge regression.

Findings

Enhanced interpretability and predictive accuracy in simulations.

Effective variable selection with smooth and sparse solutions.

Successful application to spectroscopy data for classification and regression.

Abstract

With a focus on linear models with smooth functional covariates, we propose a penalization framework (SACR) based on the nonzero centered ridge, where the center of the penalty is optimally reweighted in a supervised way, starting from the ordinary ridge solution as the initial centerfunction. In particular, we introduce a convex formulation that jointly estimates the model's coefficients and the weight function, with a roughness penalty on the centerfunction and constraints on the weights in order to recover a possibly smooth and/or sparse solution. This allows for a non-iterative and continuous variable selection mechanism, as the weight function can either inflate or deflate the initial center, in order to target the penalty towards a suitable center, with the objective to reduce the unwanted shrinkage on the nonzero coefficients, instead of uniformly shrinking the whole coefficient function. As empirical evidence of the interpretability and predictive power of our method, we provide a simulation study and two real world spectroscopy applications with both classification and regression.