Refined Convergence Rates for Maximum Likelihood Estimation under Finite Mixture Models

TL;DR

This paper introduces new loss functions that improve the understanding of convergence rates for maximum likelihood estimators in finite mixture models, especially when penalization is used to handle vanishing weights.

Contribution

It proposes stronger loss functions that better capture heterogeneity in convergence rates, sharpening existing theoretical bounds and extending some results to traditional MLE.

Findings

Penalized MLE components often converge faster than previously thought.

New loss functions resolve limitations of Wasserstein distance in mixture models.

Simulation confirms improved convergence rate estimates.

Abstract

We revisit the classical problem of deriving convergence rates for the maximum likelihood estimator (MLE) in finite mixture models. The Wasserstein distance has become a standard loss function for the analysis of parameter estimation in these models, due in part to its ability to circumvent label switching and to accurately characterize the behaviour of fitted mixture components with vanishing weights. However, the Wasserstein distance is only able to capture the worst-case convergence rate among the remaining fitted mixture components. We demonstrate that when the log-likelihood function is penalized to discourage vanishing mixing weights, stronger loss functions can be derived to resolve this shortcoming of the Wasserstein distance. These new loss functions accurately capture the heterogeneity in convergence rates of fitted mixture components, and we use them to sharpen existing pointwise and uniform convergence rates in various classes of mixture models. In particular, these results imply that a subset of the components of the penalized MLE typically converge significantly faster than could have been anticipated from past work. We further show that some of these conclusions extend to the traditional MLE. Our theoretical findings are supported by a simulation study to illustrate these improved convergence rates.