A Proximal Point Algorithm for Minimum Divergence Estimators with Application to Mixture Models

TL;DR

This paper introduces a proximal point algorithm for computing minimum divergence estimators, specifically MDϕDE, with applications to mixture models, offering improved convergence properties and robustness over traditional methods like EM.

Contribution

The paper develops a new proximal point algorithm for MDϕDE estimators, relaxing identifiability conditions and analyzing convergence for mixture models, with practical simulation validation.

Findings

Algorithm converges under relaxed conditions

Enhanced robustness of estimators against outliers

Validated with simulations on mixture models

Abstract

Estimators derived from a divergence criterion such as $\varphi-$divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called MD$\varphi$DE, an estimator built using a dual representation of $\varphi$--divergences. We present in this paper an iterative proximal point algorithm which permits to calculate such estimator. This algorithm contains by its construction the well-known EM algorithm. Our work is based on the paper of \citep{Tseng} on the likelihood function. We provide several convergence properties of the sequence generated by the algorithm, and improve the existing results by relaxing the identifiability condition on the proximal term, a condition which is not verified for most mixture models and hard to be verified for non mixture ones. Since convergence analysis uses regularity conditions (continuity and differentiability) of the objective function, which has a supremal form, we find it useful to present some analytical approaches for studying such functions. Convergence of the EM algorithm is discussed here again in a Gaussian and Weibull mixtures in the spirit of our approach. Simulations are provided to confirm the validity of our work and the robustness of the resulting estimators against outliers.