Dendrogram of mixing measures: Hierarchical clustering and model selection for finite mixture models

TL;DR

This paper introduces a novel hierarchical clustering approach based on dendrograms derived from overfitted mixture models, enabling consistent model selection and detailed subpopulation insights, supported by theory and simulations.

Contribution

It develops a new method linking hierarchical clustering with mixture model selection, providing theoretical guarantees and practical advantages over traditional summarization techniques.

Findings

Consistent selection of the true number of mixture components.

Optimal convergence rates for parameter estimation.

Enhanced hierarchical subpopulation insights in applications.

Abstract

We present a new way to summarize and select mixture models via the hierarchical clustering tree (dendrogram) constructed from an overfitted latent mixing measure. Our proposed method bridges agglomerative hierarchical clustering and mixture modeling. The dendrogram's construction is derived from the theory of convergence of the mixing measures, and as a result, we can both consistently select the true number of mixing components and obtain the pointwise optimal convergence rate for parameter estimation from the tree, even when the model parameters are only weakly identifiable. In theory, it explicates the choice of the optimal number of clusters in hierarchical clustering. In practice, the dendrogram reveals more information on the hierarchy of subpopulations compared to traditional ways of summarizing mixture models. Several simulation studies are carried out to support our theory. We also illustrate the methodology with an application to single-cell RNA sequence analysis.