# Mini-batch learning of exponential family finite mixture models

## TL;DR

This paper introduces a mini-batch EM algorithm framework for exponential family mixture models, providing convergence theory, and demonstrating improved performance over standard EM in simulations and real data applications.

## Contribution

It develops a novel mini-batch EM algorithm with stochastic stabilization for exponential family mixtures, including theoretical convergence results and practical performance improvements.

## Key findings

- Mini-batch EM algorithms converge under certain conditions.
- The mini-batch approach outperforms standard EM in normal mixture estimation.
- Application to MNIST data shows practical effectiveness.

## Abstract

Mini-batch algorithms have become increasingly popular due to the requirement for solving optimization problems, based on large-scale data sets. Using an existing online expectation-{}-maximization (EM) algorithm framework, we demonstrate how mini-batch (MB) algorithms may be constructed, and propose a scheme for the stochastic stabilization of the constructed mini-batch algorithms. Theoretical results regarding the convergence of the mini-batch EM algorithms are presented. We then demonstrate how the mini-batch framework may be applied to conduct maximum likelihood (ML) estimation of mixtures of exponential family distributions, with emphasis on ML estimation for mixtures of normal distributions. Via a simulation study, we demonstrate that the mini-batch algorithm for mixtures of normal distributions can outperform the standard EM algorithm. Further evidence of the performance of the mini-batch framework is provided via an application to the famous MNIST data set.