A Semiparametric Nonlinear Mixed Effects Model with Penalized Splines Using Automatic Differentiation

TL;DR

This paper introduces a novel estimation method for nonlinear mixed-effects models using penalized splines and automatic differentiation, improving inference accuracy and computational efficiency.

Contribution

It develops a semiparametric modeling approach with joint smoothness estimation and automatic differentiation, enhancing existing nonlinear mixed-effects modeling techniques.

Findings

Improved inferential performance in simulations

Reduced computational burden compared to existing methods

Effective modeling of infant height growth

Abstract

We present an estimation procedure for nonlinear mixed-effects models in which the population trajectory is represented by penalized splines and adapted to individuals via subject-specific transformation parameters. By exploiting the mixed model representation of penalized splines, the level of smoothness can be estimated jointly with other variance components. The integration over random effects needed to obtain the marginal likelihood is carried out using the Laplace approximation. Exact derivatives for evaluation and maximization of the resulting likelihood are obtained via automatic differentiation implemented through Template Model Builder. In simulation studies, the method produces improved inferential performance and reduced computational burden when compared to the existing procedure. The approach is further illustrated through a case study on infant height growth in the first two years of life.