Systematically and efficiently improving $k$-means initialization by pairwise-nearest-neighbor smoothing

TL;DR

The paper introduces PNN-smoothing, a meta-method that improves $k$-means initialization by combining subset clustering with pairwise-nearest-neighbor merging, leading to better and faster clustering results.

Contribution

It proposes a novel meta-method for $k$-means seeding that enhances existing algorithms through subset clustering and PNN merging, improving efficiency and effectiveness.

Findings

PNN-smoothing consistently improves clustering costs across datasets.

Enhancement of $k$-means++ seeding with PNN-smoothing outperforms greedy $k$-means++ in speed and quality.

The method is computationally efficient, nearly linear with data size for suitable parameters.

Abstract

We present a meta-method for initializing (seeding) the $k$-means clustering algorithm called PNN-smoothing. It consists in splitting a given dataset into $J$ random subsets, clustering each of them individually, and merging the resulting clusterings with the pairwise-nearest-neighbor (PNN) method. It is a meta-method in the sense that when clustering the individual subsets any seeding algorithm can be used. If the computational complexity of that seeding algorithm is linear in the size of the data $N$ and the number of clusters $k$, PNN-smoothing is also almost linear with an appropriate choice of $J$, and quite competitive in practice. We show empirically, using several existing seeding methods and testing on several synthetic and real datasets, that this procedure results in systematically better costs. In particular, our method of enhancing $k$-means++ seeding proves superior in both effectiveness and speed compared to the popular "greedy" $k$-means++ variant. Our implementation is publicly available at https://github.com/carlobaldassi/KMeansPNNSmoothing.jl.