A Unified Framework for Center-based Clustering of Distributed Data

TL;DR

This paper introduces a unified distributed clustering framework that enables users with local data to collaboratively find a global clustering solution over a network, applicable to various loss functions including K-means.

Contribution

The paper proposes a novel family of distributed clustering algorithms, DGC-𝓕_ρ, with a unified analysis, applicable to multiple loss functions, and guarantees convergence to meaningful fixed points.

Findings

Centers converge to fixed points under mild conditions.

As ρ increases, fixed points approach consensus.

Algorithms perform well on synthetic and real data.

Abstract

We develop a family of distributed center-based clustering algorithms that work over networks of users. In the proposed scenario, users contain a local dataset and communicate only with their immediate neighbours, with the aim of finding a clustering of the full, joint data. The proposed family, termed Distributed Gradient Clustering (DGC-$\mathcal{F}_\rho$), is parametrized by $\rho \geq 1$, controling the proximity of users' center estimates, with $\mathcal{F}$ determining the clustering loss. Our framework allows for a broad class of smooth convex loss functions, including popular clustering losses like $K$-means and Huber loss. Specialized to popular clustering losses like $K$-means and Huber loss, DGC-$\mathcal{F}_\rho$ gives rise to novel distributed clustering algorithms DGC-KM$_\rho$ and DGC-HL$_\rho$, while novel clustering losses based on Logistic and Fair functions lead to DGC-LL$_\rho$ and DGC-FL$_\rho$. We provide a unified analysis and establish several strong results, under mild assumptions. First, we show that the sequence of centers generated by the methods converges to a well-defined notion of fixed point, under any center initialization and value of $\rho$. Second, we prove that, as $\rho$ increases, the family of fixed points produced by DGC-$\mathcal{F}_\rho$ converges to a notion of consensus fixed points. We show that consensus fixed points of DGC-$\mathcal{F}_{\rho}$ are equivalent to fixed points of gradient clustering over the full data, guaranteeing a clustering of the full data is produced. For the special case of Bregman losses, we show that our fixed points converge to the set of Lloyd points. Extensive numerical experiments on synthetic and real data confirm our theoretical findings, show strong performance of our methods and demonstrate the usefulness and wide range of potential applications of our general framework, such as outlier detection.