Identifiability and optimal rates of convergence for parameters of multiple types in finite mixtures

TL;DR

This paper develops a comprehensive theory for the identifiability and convergence rates of parameters in finite mixture models, highlighting differences between well-behaved and weakly identifiable classes, with implications for model fitting and estimation accuracy.

Contribution

It extends strong identifiability theory to matrix-variate parameters and characterizes convergence rates for various mixture models, including those with complex algebraic structures.

Findings

Optimal convergence rates are $n^{-1/2}$ under $W_1$ for exact-fitted models.

Over-fitted models exhibit slower rates, such as $n^{-1/4}$ under $W_2$.

Algebraic structures determine the estimation rates in weakly identifiable models.

Abstract

This paper studies identifiability and convergence behaviors for parameters of multiple types in finite mixtures, and the effects of model fitting with extra mixing components. First, we present a general theory for strong identifiability, which extends from the previous work of Nguyen [2013] and Chen [1995] to address a broad range of mixture models and to handle matrix-variate parameters. These models are shown to share the same Wasserstein distance based optimal rates of convergence for the space of mixing distributions --- $n^{-1/2}$ under $W_1$ for the exact-fitted and $n^{-1/4}$ under $W_2$ for the over-fitted setting, where $n$ is the sample size. This theory, however, is not applicable to several important model classes, including location-scale multivariate Gaussian mixtures, shape-scale Gamma mixtures and location-scale-shape skew-normal mixtures. The second part of this work is devoted to demonstrating that for these "weakly identifiable" classes, algebraic structures of the density family play a fundamental role in determining convergence rates of the model parameters, which display a very rich spectrum of behaviors. For instance, the optimal rate of parameter estimation in an over-fitted location-covariance Gaussian mixture is precisely determined by the order of a solvable system of polynomial equations --- these rates deteriorate rapidly as more extra components are added to the model. The established rates for a variety of settings are illustrated by a simulation study.