On the Semi-supervised Expectation Maximization

TL;DR

This paper analyzes how labeled samples influence the convergence rate of the EM algorithm in semi-supervised learning, providing theoretical guarantees and extending results for Gaussian mixture models.

Contribution

It offers a convergence rate analysis for semi-supervised EM, highlighting the role of labeled data, and extends findings to Gaussian mixtures with theoretical proofs.

Findings

Labeled samples improve convergence rate for exponential family mixtures

Provides a comprehensive convergence analysis for Gaussian mixture models

Extends proof for symmetric Gaussian mixtures with unlabeled data

Abstract

The Expectation Maximization (EM) algorithm is widely used as an iterative modification to maximum likelihood estimation when the data is incomplete. We focus on a semi-supervised case to learn the model from labeled and unlabeled samples. Existing work in the semi-supervised case has focused mainly on performance rather than convergence guarantee, however we focus on the contribution of the labeled samples to the convergence rate. The analysis clearly demonstrates how the labeled samples improve the convergence rate for the exponential family mixture model. In this case, we assume that the population EM (EM with unlimited data) is initialized within the neighborhood of global convergence for the population EM that consists solely of samples that have not been labeled. The analysis for the labeled samples provides a comprehensive description of the convergence rate for the Gaussian mixture model. In addition, we extend the findings for labeled samples and offer an alternative proof for the population EM's convergence rate with unlabeled samples for the symmetric mixture of two Gaussians.