A SAEM Algorithm for Fused Lasso Penalized Non Linear Mixed Effect Models: Application to Group Comparison in Pharmacokinetic

TL;DR

This paper introduces a novel penalized likelihood method using a SAEM algorithm with fused lasso penalty to compare multiple groups in nonlinear mixed effect models, specifically applied to pharmacokinetic data.

Contribution

It develops a new penalized likelihood approach with a specialized SAEM algorithm for group comparison in nonlinear mixed models, allowing for multiple group analysis beyond the traditional reference group method.

Findings

The algorithm effectively compares more than two groups in simulations.

Application to pharmacokinetic data demonstrates practical utility.

Fused lasso penalty induces sparsity in group differences.

Abstract

Non linear mixed effect models are classical tools to analyze non linear longitudinal data in many fields such as population Pharmacokinetic. Groups of observations are usually compared by introducing the group affiliations as binary covariates with a reference group that is stated among the groups. This approach is relatively limited as it allows only the comparison of the reference group to the others. In this work, we propose to compare the groups using a penalized likelihood approach. Groups are described by the same structural model but with parameters that are group specific. The likelihood is penalized with a fused lasso penalty that induces sparsity on the differences between groups for both fixed effects and variances of random effects. A penalized Stochastic Approximation EM algorithm is proposed that is coupled to Alternating Direction Method Multipliers to solve the maximization step. An extensive simulation study illustrates the performance of this algorithm when comparing more than two groups. Then the approach is applied to real data from two pharmacokinetic drug-drug interaction trials.