Rates of Convergence of Maximum Smoothed Log-Likelihood Estimators for Semi-Parametric Multivariate Mixtures

TL;DR

This paper establishes theoretical guarantees and convergence rates for a smoothed likelihood estimator in semi-parametric multivariate mixture models, enhancing understanding of its statistical properties.

Contribution

It provides the first rigorous theoretical analysis of a smoothed likelihood estimator in semi-parametric mixture models, including convergence rates and consistency.

Findings

Proves consistency of the estimator under mild conditions

Derives convergence rates for finite- and infinite-dimensional parameters

Bridges practical algorithms with theoretical guarantees in semi-parametric modeling

Abstract

Theoretical guarantees are established for a standard estimator in a semi-parametric finite mixture model, where each component density is modeled as a product of univariate densities under a conditional independence assumption. The focus is on the estimator that maximizes a smoothed log-likelihood function, which can be efficiently computed using a majorization-minimization algorithm. This smoothed likelihood applies a nonlinear regularization operator defined as the exponential of a kernel convolution on the logarithm of each component density. Consistency of the estimators is demonstrated by leveraging classical M-estimation frameworks under mild regularity conditions. Subsequently, convergence rates for both finite- and infinite-dimensional parameters are derived by exploiting structural properties of the smoothed likelihood, the behavior of the iterative optimization algorithm, and a thorough study of the profile smoothed likelihood. This work provides the first rigorous theoretical guarantees for this estimation approach, bridging the gap between practical algorithms and statistical theory in semi-parametric mixture modeling.