Mixed Effects Mixture of Experts: Modeling Double Heterogeneous Trajectories

TL;DR

This paper introduces MEMoE, a novel statistical framework combining mixed-effects models with mixture of experts to better analyze heterogeneous longitudinal data, improving accuracy and interpretability.

Contribution

The paper proposes MEMoE, integrating LMMs with Mixture of Experts, and develops a robust estimation procedure for modeling double heterogeneity in trajectories.

Findings

MEMoE outperforms traditional LMMs and Mixture of Experts in simulations.

MEMoE achieves higher classification accuracy and better model fit.

Robust inference procedure effectively estimates parameters even with model misspecification.

Abstract

Linear mixed-effects model (LMM) is a cornerstone of longitudinal data analysis, but is limited to adeptly make heterogeneous analyses predictable under both group-specific fixed effects and subject-specific random effects. To address this challenge, we propose a novel statistical framework by using a large model prototype: a mixed effects mixture of experts model (MEMoE). This framework integrates the divide-and-conquer paradigm of Mixture of Experts Models with classical mixed-effect modeling. In the proposed MEMoE, each expert is a full LMM dedicated to capturing the longitudinal trajectory of a specific latent subpopulation, while another model gating function learns to route subjects to the most appropriate expert in a data-driven manner based on baseline covariates. We develop a robust inferential procedure for parameter estimation based on the Laplace Expectation-Maximization algorithm, with standard errors calibrated using robust sandwich estimators to account for potential model misspecification. Extensive simulation studies and an empirical application demonstrate that MEMoE outperforms both traditional single-population LMM and conventional Mixture of Experts models in terms of parameter recovery, classification accuracy, and overall model fit.