A description length approach to determining the number of k-means clusters

TL;DR

This paper introduces an asymptotic criterion based on description length to determine the optimal number of clusters in k-means, treating clustering as data compression and evaluating hierarchical structure through multi-stage compression.

Contribution

It proposes a novel description length-based method for selecting the number of k-means clusters, including criteria for hierarchical structure assessment.

Findings

Method provides reasonable clustering results for dimension reduction.

Criteria depend on dataset properties like dimensionality.

Approach offers a principled way to determine clusters when data properties are unknown.

Abstract

We present an asymptotic criterion to determine the optimal number of clusters in k-means. We consider k-means as data compression, and propose to adopt the number of clusters that minimizes the estimated description length after compression. Here we report two types of compression ratio based on two ways to quantify the description length of data after compression. This approach further offers a way to evaluate whether clusters obtained with k-means have a hierarchical structure by examining whether multi-stage compression can further reduce the description length. We applied our criteria to determine the number of clusters to synthetic data and empirical neuroimaging data to observe the behavior of the criteria across different types of data set and suitability of the two types of criteria for different datasets. We found that our method can offer reasonable clustering results that are useful for dimension reduction. While our numerical results revealed dependency of our criteria on the various aspects of dataset such as the dimensionality, the description length approach proposed here provides a useful guidance to determine the number of clusters in a principled manner when underlying properties of the data are unknown and only inferred from observation of data.