Imputation Maximization Stochastic Approximation with Application to Generalized Linear Mixed Models

TL;DR

This paper introduces IMSA, a novel stochastic approximation method for generalized linear mixed models that improves estimation stability, reduces bias, and outperforms existing methods like ScoreSA in simulations.

Contribution

The paper proposes IMSA, a new imputation-based stochastic approximation algorithm for generalized linear mixed models, addressing likelihood intractability and enhancing estimation accuracy.

Findings

IMSA achieves more stable convergence than ScoreSA.

IMSA provides less biased estimates in finite samples.

Simulation studies show IMSA outperforms ScoreSA across various scenarios.

Abstract

Generalized linear mixed models are useful in studying hierarchical data with possibly non-Gaussian responses. However, the intractability of likelihood functions poses challenges for estimation. We develop a new method suitable for this problem, called imputation maximization stochastic approximation (IMSA). For each iteration, IMSA first imputes latent variables/random effects, then maximizes over the complete data likelihood, and finally moves the estimate towards the new maximizer while preserving a proportion of the previous value. The limiting point of IMSA satisfies a self-consistency property and can be less biased in finite samples than the maximum likelihood estimator solved by score-equation based stochastic approximation (ScoreSA). Numerically, IMSA can also be advantageous over ScoreSA in achieving more stable convergence and respecting the parameter ranges under various transformations such as nonnegative variance components. This is corroborated through our simulation studies where IMSA consistently outperforms ScoreSA.