$K$-Means and Gaussian Mixture Modeling with a Separation Constraint

TL;DR

This paper introduces a novel approach to clustering with $K$-means and Gaussian mixture models by incorporating a separation constraint on cluster centers, using dynamic programming and EM algorithms to improve regularization and clustering quality.

Contribution

It presents the first algorithms for $K$-means and Gaussian mixture models with separation constraints, enhancing clustering regularization and solution interpretability.

Findings

Separation constraints improve clustering regularization.

The proposed algorithms perform well on simulated and real data.

Separation constraints lead to more distinct and meaningful clusters.

Abstract

We consider the problem of clustering with $K$-means and Gaussian mixture models with a constraint on the separation between the centers in the context of real-valued data. We first propose a dynamic programming approach to solving the $K$-means problem with a separation constraint on the centers, building on (Wang and Song, 2011). In the context of fitting a Gaussian mixture model, we then propose an EM algorithm that incorporates such a constraint. A separation constraint can help regularize the output of a clustering algorithm, and we provide both simulated and real data examples to illustrate this point.