K-groups: A Generalization of K-means Clustering

TL;DR

This paper introduces k-groups, a distribution-based clustering method using energy distance, which generalizes k-means and performs better on skewed, heavy-tailed, or high-dimensional data.

Contribution

The paper proposes a novel class of clustering algorithms based on energy distance, extending k-means to handle non-spherical, skewed, and heavy-tailed distributions.

Findings

k-groups performs as well as k-means on well-separated, normal data.

k-groups outperforms k-means on skewed, heavy-tailed, and high-dimensional data.

k-groups by first variation is consistent as dimension increases.

Abstract

We propose a new class of distribution-based clustering algorithms, called k-groups, based on energy distance between samples. The energy distance clustering criterion assigns observations to clusters according to a multi-sample energy statistic that measures the distance between distributions. The energy distance determines a consistent test for equality of distributions, and it is based on a population distance that characterizes equality of distributions. The k-groups procedure therefore generalizes the k-means method, which separates clusters that have different means. We propose two k-groups algorithms: k-groups by first variation; and k-groups by second variation. The implementation of k-groups is partly based on Hartigan and Wong's algorithm for k-means. The algorithm is generalized from moving one point on each iteration (first variation) to moving $m$ $(m > 1)$ points. For univariate data, we prove that Hartigan and Wong's k-means algorithm is a special case of k-groups by first variation. The simulation results from univariate and multivariate cases show that our k-groups algorithms perform as well as Hartigan and Wong's k-means algorithm when clusters are well-separated and normally distributed. Moreover, both k-groups algorithms perform better than k-means when data does not have a finite first moment or data has strong skewness and heavy tails. For non--spherical clusters, both k-groups algorithms performed better than k-means in high dimension, and k-groups by first variation is consistent as dimension increases. In a case study on dermatology data with 34 features, both k-groups algorithms performed better than k-means.