Mixture-Models: a one-stop Python Library for Model-based Clustering using various Mixture Models

TL;DR

Mixture-Models is a comprehensive Python library that simplifies fitting and analyzing various mixture models, including high-dimensional data, with automatic differentiation and multiple optimization routines.

Contribution

It introduces a versatile, extensible Python library for model-based clustering using various mixture models with advanced optimization and evaluation tools.

Findings

Gradient-based methods outperform EM in many settings.

Library supports high-dimensional data modeling.

Extensible framework for new distributions and optimization techniques.

Abstract

\texttt{Mixture-Models} is an open-source Python library for fitting Gaussian Mixture Models (GMM) and their variants, such as Parsimonious GMMs, Mixture of Factor Analyzers, MClust models, Mixture of Student's t distributions, etc. It streamlines the implementation and analysis of these models using various first/second order optimization routines such as Gradient Descent and Newton-CG through automatic differentiation (AD) tools. This helps in extending these models to high-dimensional data, which is first of its kind among Python libraries. The library provides user-friendly model evaluation tools, such as BIC, AIC, and log-likelihood estimation. The source-code is licensed under MIT license and can be accessed at \url{https://github.com/kasakh/Mixture-Models}. The package is highly extensible, allowing users to incorporate new distributions and optimization techniques with ease. We conduct a large scale simulation to compare the performance of various gradient based approaches against Expectation Maximization on a wide range of settings and identify the corresponding best suited approach.