Scalable and Incremental Learning of Gaussian Mixture Models

TL;DR

This paper introduces a fast, scalable incremental algorithm for Gaussian mixture models that efficiently updates parameters, enabling high-dimensional data processing and applications in classification, function approximation, and control tasks.

Contribution

It proposes a novel incremental learning algorithm for Gaussian mixture models with rank-one updates, improving scalability and efficiency for high-dimensional data.

Findings

Algorithm achieves igO{NKD^2} complexity

Effective on high-dimensional datasets like MNIST and CIFAR-10

Demonstrates applicability in reinforcement learning tasks

Abstract

This work presents a fast and scalable algorithm for incremental learning of Gaussian mixture models. By performing rank-one updates on its precision matrices and determinants, its asymptotic time complexity is of \BigO{NKD^2} for $N$ data points, $K$ Gaussian components and $D$ dimensions. The resulting algorithm can be applied to high dimensional tasks, and this is confirmed by applying it to the classification datasets MNIST and CIFAR-10. Additionally, in order to show the algorithm's applicability to function approximation and control tasks, it is applied to three reinforcement learning tasks and its data-efficiency is evaluated.