Smoothly varying ridge regularization

TL;DR

This paper introduces a new smoothly varying regularization method with adaptive penalties for nonlinear regression, effectively handling functions with inhomogeneous smoothness while reducing computational load.

Contribution

The paper proposes a novel efficient basis expansion method using adaptive-type penalties and derives an approximate GIC for model selection, improving estimation for complex functions.

Findings

Method performs well in Monte Carlo simulations.

Numerical results show effectiveness in real data analysis.

Reduces computational complexity compared to existing adaptive methods.

Abstract

A basis expansion with regularization methods is much appealing to the flexible or robust nonlinear regression models for data with complex structures. When the underlying function has inhomogeneous smoothness, it is well known that conventional reguralization methods do not perform well. In this case, an adaptive procedure such as a free-knot spline or a local likelihood method is often introduced as an effective method. However, both methods need intensive computational loads. In this study, we consider a new efficient basis expansion by proposing a smoothly varying regularization method which is constructed by some special penalties. We call them adaptive-type penalties. In our modeling, adaptive-type penalties play key rolls and it has been successful in giving good estimation for inhomogeneous smoothness functions. A crucial issue in the modeling process is the choice of a suitable model among candidates. To select the suitable model, we derive an approximated generalized information criterion (GIC). The proposed method is investigated through Monte Carlo simulations and real data analysis. Numerical results suggest that our method performs well in various situations.