High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality

TL;DR

This paper develops a high-dimensional EM algorithm that incorporates sparsity, achieving optimal estimation and inference for latent variable models with strong theoretical guarantees and practical performance.

Contribution

It introduces a novel high-dimensional EM algorithm with convergence guarantees and inference procedures for low-dimensional components, advancing statistical optimization in high dimensions.

Findings

Algorithm converges at a geometric rate with proper initialization.

Estimator attains near-optimal statistical convergence rates.

Framework enables computationally feasible inference in high-dimensional models.

Abstract

We provide a general theory of the expectation-maximization (EM) algorithm for inferring high dimensional latent variable models. In particular, we make two contributions: (i) For parameter estimation, we propose a novel high dimensional EM algorithm which naturally incorporates sparsity structure into parameter estimation. With an appropriate initialization, this algorithm converges at a geometric rate and attains an estimator with the (near-)optimal statistical rate of convergence. (ii) Based on the obtained estimator, we propose new inferential procedures for testing hypotheses and constructing confidence intervals for low dimensional components of high dimensional parameters. For a broad family of statistical models, our framework establishes the first computationally feasible approach for optimal estimation and asymptotic inference in high dimensions. Our theory is supported by thorough numerical results.