Group-wise shrinkage estimation in penalized model-based clustering

TL;DR

This paper introduces a data-driven, group-wise penalization method for Gaussian mixture models to improve clustering in high-dimensional data by accounting for varying variable associations across groups.

Contribution

It develops a novel group-wise penalty approach that automatically adjusts for different levels of variable connectivity in model-based clustering, without extra hyper-parameters.

Findings

Effective in high-dimensional settings with synthetic and real data.

Automatically adapts to different sparsity levels across groups.

Improves clustering accuracy by modeling variable associations more flexibly.

Abstract

Finite Gaussian mixture models provide a powerful and widely employed probabilistic approach for clustering multivariate continuous data. However, the practical usefulness of these models is jeopardized in high-dimensional spaces, where they tend to be over-parameterized. As a consequence, different solutions have been proposed, often relying on matrix decompositions or variable selection strategies. Recently, a methodological link between Gaussian graphical models and finite mixtures has been established, paving the way for penalized model-based clustering in the presence of large precision matrices. Notwithstanding, current methodologies implicitly assume similar levels of sparsity across the classes, not accounting for different degrees of association between the variables across groups. We overcome this limitation by deriving group-wise penalty factors, which automatically enforce under or over-connectivity in the estimated graphs. The approach is entirely data-driven and does not require additional hyper-parameter specification. Analyses on synthetic and real data showcase the validity of our proposal.