Wasserstein $K$-means for clustering probability distributions

TL;DR

This paper introduces a Wasserstein $K$-means clustering method for probability distributions, demonstrating its advantages over centroid-based approaches, especially in the context of Gaussian distributions and real-world data.

Contribution

The paper proposes a distance-based Wasserstein $K$-means with SDP relaxation, showing it can recover true clusters and outperform centroid-based methods in clustering distributions.

Findings

SDP relaxation achieves exact recovery for well-separated Gaussian clusters.

Distance-based Wasserstein $K$-means outperforms centroid-based methods in experiments.

Wasserstein barycenters face regularity issues affecting centroid-based clustering.

Abstract

Clustering is an important exploratory data analysis technique to group objects based on their similarity. The widely used $K$-means clustering method relies on some notion of distance to partition data into a fewer number of groups. In the Euclidean space, centroid-based and distance-based formulations of the $K$-means are equivalent. In modern machine learning applications, data often arise as probability distributions and a natural generalization to handle measure-valued data is to use the optimal transport metric. Due to non-negative Alexandrov curvature of the Wasserstein space, barycenters suffer from regularity and non-robustness issues. The peculiar behaviors of Wasserstein barycenters may make the centroid-based formulation fail to represent the within-cluster data points, while the more direct distance-based $K$-means approach and its semidefinite program (SDP) relaxation are capable of recovering the true cluster labels. In the special case of clustering Gaussian distributions, we show that the SDP relaxed Wasserstein $K$-means can achieve exact recovery given the clusters are well-separated under the $2$-Wasserstein metric. Our simulation and real data examples also demonstrate that distance-based $K$-means can achieve better classification performance over the standard centroid-based $K$-means for clustering probability distributions and images.