A Component-wise EM Algorithm for Mixtures

TL;DR

This paper introduces a component-wise EM algorithm for finite mixture estimation that updates parameters sequentially, improving convergence and providing a theoretical convergence proof, with numerical comparisons to existing methods.

Contribution

It proposes a novel component-wise EM algorithm for mixtures, with convergence analysis and empirical performance evaluation.

Findings

The component-wise EM converges reliably, unlike standard EM in some cases.

Numerical experiments demonstrate improved convergence speed.

The method compares favorably to SAGE algorithm in experiments.

Abstract

In some situations, EM algorithm shows slow convergence problems. One possible reason is that standard procedures update the parameters simultaneously. In this paper we focus on finite mixture estimation. In this framework, we propose a component-wise EM, which updates the parameters sequentially. We give an interpretation of this procedure as a proximal point algorithm and use it to prove the convergence. Illustrative numerical experiments show how our algorithm compares to EM and a version of the SAGE algorithm.