Fast convergence of the Expectation Maximization algorithm under a logarithmic Sobolev inequality

TL;DR

This paper introduces a new analytical framework for the EM algorithm using gradient flow theory, establishing exponential convergence and error bounds under a generalized log-Sobolev inequality, applicable to various EM variants.

Contribution

It extends gradient flow techniques to analyze EM, providing finite sample bounds and convergence guarantees under a generalized log-Sobolev inequality, unifying analysis of EM variants.

Findings

Exponential convergence of EM under the new framework

Finite sample error bounds for EM

Unified analysis applicable to EM variants

Abstract

We present a new framework for analysing the Expectation Maximization (EM) algorithm. Drawing on recent advances in the theory of gradient flows over Euclidean-Wasserstein spaces, we extend techniques from alternating minimization in Euclidean spaces to the EM algorithm, via its representation as coordinate-wise minimization of the free energy. In so doing, we obtain finite sample error bounds and exponential convergence of the EM algorithm under a natural generalisation of the log-Sobolev inequality. We further show that this framework naturally extends to several variants of EM, offering a unified approach for studying such algorithms.