A Riemannian Newton Trust-Region Method for Fitting Gaussian Mixture Models

TL;DR

This paper introduces a Riemannian Newton Trust-Region method for Gaussian Mixture Models that improves convergence speed and efficiency, especially in complex clustering scenarios with hidden data.

Contribution

It provides an explicit Riemannian Hessian formula and a novel optimization algorithm that outperform existing methods in runtime and iteration count.

Findings

Outperforms current approaches in runtime and iterations

Effective for data with high hidden information

Applicable to clustering and density estimation tasks

Abstract

Gaussian Mixture Models are a powerful tool in Data Science and Statistics that are mainly used for clustering and density approximation. The task of estimating the model parameters is in practice often solved by the Expectation Maximization (EM) algorithm which has its benefits in its simplicity and low per-iteration costs. However, the EM converges slowly if there is a large share of hidden information or overlapping clusters. Recent advances in manifold optimization for Gaussian Mixture Models have gained increasing interest. We introduce an explicit formula for the Riemannian Hessian for Gaussian Mixture Models. On top, we propose a new Riemannian Newton Trust-Region method which outperforms current approaches both in terms of runtime and number of iterations. We apply our method on clustering problems and density approximation tasks. Our method is very powerful for data with a large share of hidden information compared to existing methods.