Clustering and variable selection for categorical multivariate data

TL;DR

This paper develops a new model selection method for clustering categorical multivariate data using multinomial mixture models, with theoretical guarantees and improved practical performance over classical criteria like BIC and AIC.

Contribution

It introduces a novel penalized maximum likelihood criterion with a data-driven penalty calibration, ensuring asymptotic consistency for variable selection in categorical data clustering.

Findings

The new criterion outperforms BIC and AIC in simulations.

The method is asymptotically consistent under weak assumptions.

Practical application demonstrates improved model selection accuracy.

Abstract

This article investigates unsupervised classification techniques for categorical multivariate data. The study employs multivariate multinomial mixture modeling, which is a type of model particularly applicable to multilocus genotypic data. A model selection procedure is used to simultaneously select the number of components and the relevant variables. A non-asymptotic oracle inequality is obtained, leading to the proposal of a new penalized maximum likelihood criterion. The selected model proves to be asymptotically consistent under weak assumptions on the true probability underlying the observations. The main theoretical result obtained in this study suggests a penalty function defined to within a multiplicative parameter. In practice, the data-driven calibration of the penalty function is made possible by slope heuristics. Based on simulated data, this procedure is found to improve the performance of the selection procedure with respect to classical criteria such as BIC and AIC. The new criterion provides an answer to the question "Which criterion for which sample size?" Examples of real dataset applications are also provided.