Online and Distributed learning of Gaussian mixture models by Bayesian Moment Matching

TL;DR

This paper introduces a Bayesian moment matching method for online and distributed learning of Gaussian mixture models, offering improved efficiency and accuracy over traditional online EM algorithms in big data scenarios.

Contribution

It proposes a novel Bayesian approach that facilitates online and distributed parameter estimation for Gaussian mixture models by projecting the posterior onto tractable distributions.

Findings

Outperforms online EM in accuracy on data modeling benchmarks.

Reduces computational time compared to traditional methods.

Enables scalable distributed learning for large datasets.

Abstract

The Gaussian mixture model is a classic technique for clustering and data modeling that is used in numerous applications. With the rise of big data, there is a need for parameter estimation techniques that can handle streaming data and distribute the computation over several processors. While online variants of the Expectation Maximization (EM) algorithm exist, their data efficiency is reduced by a stochastic approximation of the E-step and it is not clear how to distribute the computation over multiple processors. We propose a Bayesian learning technique that lends itself naturally to online and distributed computation. Since the Bayesian posterior is not tractable, we project it onto a family of tractable distributions after each observation by matching a set of sufficient moments. This Bayesian moment matching technique compares favorably to online EM in terms of time and accuracy on a set of data modeling benchmarks.