Exponential Consistency of M-estimators in Generalized Linear Mixed Models

TL;DR

This paper establishes that M-estimators in generalized linear mixed models exhibit exponential consistency, providing theoretical guarantees for their robustness and convergence rates under data contamination.

Contribution

It proves exponential tail bounds and consistency rates for M-estimators in generalized linear mixed models, extending previous results from univariate to multivariate responses.

Findings

Exponential tail bounds for M-estimators in GLMMs

Consistency rates are established under smooth loss functions

Simulation studies confirm theoretical convergence rates

Abstract

Generalized linear mixed models are powerful tools for analyzing clustered data, where the unknown parameters are classically (and most commonly) estimated by the maximum likelihood and restricted maximum likelihood procedures. However, since the likelihood based procedures are known to be highly sensitive to outliers, M-estimators have become popular as a means to obtain robust estimates under possible data contamination. In this paper, we prove that, for sufficiently smooth general loss functions defining the M-estimators in generalized linear mixed models, the tail probability of the deviation between the estimated and the true regression coefficients have an exponential bound. This implies an exponential rate of consistency of these M-estimators under appropriate assumptions, generalizing the existing exponential consistency results from univariate to multivariate responses. We have illustrated this theoretical result further for the special examples of the maximum likelihood estimator and the robust minimum density power divergence estimator, a popular example of model-based M-estimators, in the settings of linear and logistic mixed models, comparing it with the empirical rate of convergence through simulation studies.