Fast Automatic Smoothing for Generalized Additive Models

TL;DR

This paper introduces a fast, stable, and accurate automatic smoothing method for generalized additive models (GAMs) using an empirical Bayes approach with an efficient EM algorithm, improving performance on large datasets.

Contribution

It develops a novel automatic smoothing technique for multiple GAMs that learns optimal regularization parameters efficiently via an approximate EM algorithm.

Findings

Algorithm is numerically stable and faster than existing methods.

Achieves state-of-the-art accuracy in smoothing.

Effectively applied to extremal data analysis.

Abstract

Multiple generalized additive models (GAMs) are a type of distributional regression wherein parameters of probability distributions depend on predictors through smooth functions, with selection of the degree of smoothness via $L_2$ regularization. Multiple GAMs allow finer statistical inference by incorporating explanatory information in any or all of the parameters of the distribution. Owing to their nonlinearity, flexibility and interpretability, GAMs are widely used, but reliable and fast methods for automatic smoothing in large datasets are still lacking, despite recent advances. We develop a general methodology for automatically learning the optimal degree of $L_2$ regularization for multiple GAMs using an empirical Bayes approach. The smooth functions are penalized by different amounts, which are learned simultaneously by maximization of a marginal likelihood through an approximate expectation-maximization algorithm that involves a double Laplace approximation at the E-step, and leads to an efficient M-step. Empirical analysis shows that the resulting algorithm is numerically stable, faster than all existing methods and achieves state-of-the-art accuracy. For illustration, we apply it to an important and challenging problem in the analysis of extremal data.