Number of Clusters in a Dataset: A Regularized K-means Approach

TL;DR

This paper provides theoretical bounds for the regularization parameter in k-means clustering, compares additive and multiplicative regularizers, and evaluates their effectiveness in identifying meaningful clusters in datasets.

Contribution

It introduces rigorous bounds for the regularization parameter under ideal cluster assumptions and analyzes the behavior of additive and multiplicative regularizers in k-means clustering.

Findings

Regularization bounds are derived for ideal clusters.

Additive and multiplicative regularizers can reduce solution ambiguity.

Experiments show effectiveness when clusters deviate from ideal assumptions.

Abstract

Finding the number of meaningful clusters in an unlabeled dataset is important in many applications. Regularized k-means algorithm is a possible approach frequently used to find the correct number of distinct clusters in datasets. The most common formulation of the regularization function is the additive linear term $\lambda k$, where $k$ is the number of clusters and $\lambda$ a positive coefficient. Currently, there are no principled guidelines for setting a value for the critical hyperparameter $\lambda$. In this paper, we derive rigorous bounds for $\lambda$ assuming clusters are {\em ideal}. Ideal clusters (defined as $d$-dimensional spheres with identical radii) are close proxies for k-means clusters ($d$-dimensional spherically symmetric distributions with identical standard deviations). Experiments show that the k-means algorithm with additive regularizer often yields multiple solutions. Thus, we also analyze k-means algorithm with multiplicative regularizer. The consensus among k-means solutions with additive and multiplicative regularizations reduces the ambiguity of multiple solutions in certain cases. We also present selected experiments that demonstrate performance of the regularized k-means algorithms as clusters deviate from the ideal assumption.