The MM Alternative to EM

TL;DR

This paper explains the MM algorithm as a generalization of EM, emphasizing its construction through inequalities and showcasing its effectiveness in high-dimensional optimization problems across various applications.

Contribution

It introduces the MM algorithm as a versatile alternative to EM, highlighting its construction via inequalities and demonstrating its application in complex high-dimensional problems.

Findings

MM algorithms can be constructed without missing data assumptions.

They effectively solve high-dimensional optimization problems.

Applications include random graphs, discriminant analysis, and image restoration.

Abstract

The EM algorithm is a special case of a more general algorithm called the MM algorithm. Specific MM algorithms often have nothing to do with missing data. The first M step of an MM algorithm creates a surrogate function that is optimized in the second M step. In minimization, MM stands for majorize--minimize; in maximization, it stands for minorize--maximize. This two-step process always drives the objective function in the right direction. Construction of MM algorithms relies on recognizing and manipulating inequalities rather than calculating conditional expectations. This survey walks the reader through the construction of several specific MM algorithms. The potential of the MM algorithm in solving high-dimensional optimization and estimation problems is its most attractive feature. Our applications to random graph models, discriminant analysis and image restoration showcase this ability.