Variational autoencoder for inference of nonlinear mixed effect models based on ordinary differential equations

TL;DR

This paper introduces a variational autoencoder approach for estimating parameters in nonlinear mixed-effects models based on ODEs, providing a scalable alternative to traditional MCMC-based methods especially with complex or sparse data.

Contribution

The paper presents a novel VAE-based method for parameter inference in NLME-ODEs that avoids MCMC, enabling efficient and robust estimation in complex models with irregular data.

Findings

VAE method performs comparably or better than SAEM in simulations.

The approach effectively quantifies parameter uncertainty.

Method maintains practical identifiability despite nuisance parameters.

Abstract

We propose a variational autoencoder (VAE) approach for parameter estimation in nonlinear mixed-effects models based on ordinary differential equations (NLME-ODEs) using longitudinal data from multiple subjects. In moderate dimensions, likelihood-based inference via the stochastic approximation EM algorithm (SAEM) is widely used, but it relies on Markov Chain Monte-Carlo (MCMC) to approximate subject-specific posteriors. As model complexity increases or observations per subject are sparse and irregular, performance often deteriorates due to a complex, multimodal likelihood surface which may lead to MCMC convergence difficulties. We instead estimate parameters by maximizing the evidence lower bound (ELBO), a regularized surrogate for the marginal likelihood. A VAE with a shared encoder amortizes inference of subject-specific random effects by avoiding per-subject optimization and the use of MCMC. Beyond pointwise estimation, we quantify parameter uncertainty using observed-information-based variance estimator and verify that practical identifiability of the model parameters is not compromised by nuisance parameters introduced in the encoder. We evaluate the method in three simulation case studies (pharmacokinetics, humoral response to vaccination, and TGF-$\beta$ activation dynamics in asthmatic airways) and on a real-world antibody kinetics dataset, comparing against SAEM baselines.