Parameter Expansion and Efficient Inference

TL;DR

This paper reviews parameter expansion techniques, especially PX-EM, highlighting their role in accelerating EM algorithm convergence and connecting them to efficient statistical inference.

Contribution

It provides a comprehensive review of parameter expansion, its interpretation in bias reduction, and explores its applications and implications in statistical inference and optimization.

Findings

PX-EM accelerates EM convergence without sacrificing stability

Parameter expansion links to bias reduction in inference

Potential applications in various statistical inference problems

Abstract

This EM review article focuses on parameter expansion, a simple technique introduced in the PX-EM algorithm to make EM converge faster while maintaining its simplicity and stability. The primary objective concerns the connection between parameter expansion and efficient inference. It reviews the statistical interpretation of the PX-EM algorithm, in terms of efficient inference via bias reduction, and further unfolds the PX-EM mystery by looking at PX-EM from different perspectives. In addition, it briefly discusses potential applications of parameter expansion to statistical inference and the broader impact of statistical thinking on understanding and developing other iterative optimization algorithms.