Expectation-Maximization as a Spectrally Governed Relaxation Flow

TL;DR

This paper provides a unified spectral analysis of the EM algorithm, connecting global monotonicity and local convergence through a linearized operator, leading to insights on optimal relaxation strategies.

Contribution

It explicitly characterizes the spectral properties of EM's local operator, unifying global and local convergence analysis within a single dynamical framework.

Findings

The spectrum of the local operator characterizes convergence rates.

A new optimal scalar relaxation rule for accelerated EM is derived.

Global descent and local spectral behavior are integrated into a unified analysis.

Abstract

The expectation--maximization (EM) algorithm combines global monotonicity, local linear convergence, and strong practical robustness, but these features are usually analyzed separately. Global descent is nonlinear, whereas local convergence is governed by the spectrum of the linearized EM map. How these two levels fit into a single dynamical picture has remained less transparent. We make explicit the latent-variable operator that connects them. Along the EM trajectory, the likelihood increment admits a global energy decomposition in terms of posterior-relative entropy. Linearization at a nondegenerate maximizer $\theta^\ast$ then reveals the local operator \[ \mathcal G_{\theta^\ast}=I-DT(\theta^\ast), \] which coincides with both the missing-information ratio and the information-geometric Hessian of the observed likelihood. This operator provides a unified description of local contraction, posterior rigidity, and geometric curvature. Its spectrum yields a sharp characterization of local convergence and naturally leads to an optimal scalar relaxation rule for locally accelerated EM. These results place global descent, local spectral behavior, and optimal local relaxation within a common dynamical framework.