Fitting Large Nonlinear Mixed Effects Models Using Variational Expectation Maximization

TL;DR

This paper introduces the Variational Expectation Maximization algorithm as a scalable method for fitting large nonlinear mixed effects models, demonstrating its efficiency and accuracy through computational experiments.

Contribution

It applies VEM to NLME models, showing significant scalability and efficiency improvements over traditional methods, especially for models with over 15,000 parameters.

Findings

VEM can fit NLME models with over 15,000 parameters efficiently.

VEM provides accurate results comparable to traditional methods.

Demonstrated scalability with the DeepNLME Friberg model.

Abstract

Nonlinear Mixed Effects models (NLME) models are widely used in pharmacometrics and related fields to analyze hierarchical and longitudinal data. However, as the number of parameters and random effects increases, traditional methods for maximizing the marginal likelihood become computationally expensive. This paper explores the Variational Expectation Maximization (VEM) algorithm, a scalable alternative for fitting NLME models. Originally introduced in the context of probabilistic graphical models and later popularized through variational autoencoders, VEM has not been extensively applied to NLME modeling. By leveraging flexible variational families and reverse-mode automatic differentiation, VEM can efficiently maximize the marginal likelihood, scaling to NLME models with over 15,000 population parameters. This work provides a detailed description of VEM, compares it to other NLME fitting algorithms, and highlights its scalability through computational experiments. Using the Pumas statistical software, we fit two test models: 1) a standard warfarin model, and 2) a DeepNLME Friberg model with 15,410 population parameters and 16 random effects. The warfarin model was fitted to completion to demonstrate the correctness of VEM, while the DeepNLME Friberg model was fitted for a limited number of iterations to measure the time per iteration and demonstrate VEM's scalability.