Model-based Clustering of Multi-Dimensional Zero-Inflated Counts via the EM Algorithm

TL;DR

This paper introduces a model-based clustering method for multi-dimensional zero-inflated count data using mixtures of ZIP and ZINB distributions, estimated with the EM algorithm, applicable to structured data with covariates.

Contribution

It develops a novel EM-based clustering framework for zero-inflated counts in matrix form, incorporating covariates and size factors, which advances existing methods for heterogeneous data.

Findings

High clustering accuracy demonstrated in simulations

Effective handling of covariates and size factors

Successful application to real datasets

Abstract

Zero-inflated count data arise in various fields, including health, biology, economics, and the social sciences. These data are often modelled using probabilistic distributions such as zero-inflated Poisson (ZIP), zero-inflated negative binomial (ZINB), or zero-inflated binomial (ZIB). To account for heterogeneity in the data, it is often useful to cluster observations into groups that may explain underlying differences in the data-generating process. This paper focuses on model-based clustering for zero-inflated counts when observations are structured in a matrix form rather than a vector. We propose a clustering framework based on mixtures of ZIP or ZINB distributions, with both the count and zero components depending on cluster assignments. Our approach incorporates covariates through a log-linear structure for the mean parameter and includes a size factor to adjust for differences in total sampling or exposure. Model parameters and cluster assignments are estimated via the Expectation-Maximization (EM) algorithm. We assess the performance of our proposed methodology through simulation studies evaluating clustering accuracy and estimator properties, followed by applications to publicly available datasets.