Automatic Search Intervals for the Smoothing Parameter in Penalized Splines

TL;DR

This paper introduces algorithms to automatically determine search intervals for the smoothing parameter in penalized splines, improving the robustness and efficiency of selecting optimal smoothing parameters across different criteria.

Contribution

The authors develop criterion-independent algorithms that automatically identify suitable search intervals for smoothing parameters, ensuring global optima are captured without additional computational cost.

Findings

Automatically finds search intervals containing the global optimum.

Compatible with multiple criteria like GCV and REML.

Computationally efficient and easy to implement.

Abstract

The selection of smoothing parameter is central to the estimation of penalized splines. The best value of the smoothing parameter is often the one that optimizes a smoothness selection criterion, such as generalized cross-validation error (GCV) and restricted likelihood (REML). To correctly identify the global optimum rather than being trapped in an undesired local optimum, grid search is recommended for optimization. Unfortunately, the grid search method requires a pre-specified search interval that contains the unknown global optimum, yet no guideline is available for providing this interval. As a result, practitioners have to find it by trial and error. To overcome such difficulty, we develop novel algorithms to automatically find this interval. Our automatic search interval has four advantages. (i) It specifies a smoothing parameter range where the associated penalized least squares problem is numerically solvable. (ii) It is criterion-independent so that different criteria, such as GCV and REML, can be explored on the same parameter range. (iii) It is sufficiently wide to contain the global optimum of any criterion, so that for example, the global minimum of GCV and the global maximum of REML can both be identified. (iv) It is computationally cheap compared with the grid search itself, carrying no extra computational burden in practice. Our method is ready to use through our recently developed R package gps (>= version 1.1). It may be embedded in more advanced statistical modeling methods that rely on penalized splines.