Accelerating EM: An Empirical Study

TL;DR

This paper empirically compares various acceleration techniques for the Expectation-Maximization (EM) algorithm, highlighting that the effectiveness of each method depends on specific problem properties.

Contribution

It provides a comprehensive experimental comparison of EM acceleration methods, which was previously lacking, and discusses their relative performance based on problem characteristics.

Findings

Some acceleration methods consistently improve EM convergence

The best acceleration method varies with problem properties

No single method is universally superior

Abstract

Many applications require that we learn the parameters of a model from data. EM is a method used to learn the parameters of probabilistic models for which the data for some of the variables in the models is either missing or hidden. There are instances in which this method is slow to converge. Therefore, several accelerations have been proposed to improve the method. None of the proposed acceleration methods are theoretically dominant and experimental comparisons are lacking. In this paper, we present the different proposed accelerations and try to compare them experimentally. From the results of the experiments, we argue that some acceleration of EM is always possible, but that which acceleration is superior depends on properties of the problem.