Jigsaw Game: Federated Clustering

TL;DR

This paper introduces FeCA, a one-shot federated clustering algorithm that effectively handles data heterogeneity and non-convexity, and extends it to DeepFeCA for federated unsupervised feature learning.

Contribution

The paper proposes FeCA, a novel one-shot federated clustering method, and extends it to DeepFeCA for federated unsupervised representation learning, addressing key challenges in federated clustering.

Findings

FeCA is robust across various federated scenarios.

FeCA achieves accurate clustering in a single communication round.

DeepFeCA effectively combines deep clustering with federated learning.

Abstract

Federated learning has recently garnered significant attention, especially within the domain of supervised learning. However, despite the abundance of unlabeled data on end-users, unsupervised learning problems such as clustering in the federated setting remain underexplored. In this paper, we investigate the federated clustering problem, with a focus on federated k-means. We outline the challenge posed by its non-convex objective and data heterogeneity in the federated framework. To tackle these challenges, we adopt a new perspective by studying the structures of local solutions in k-means and propose a one-shot algorithm called FeCA (Federated Centroid Aggregation). FeCA adaptively refines local solutions on clients, then aggregates these refined solutions to recover the global solution of the entire dataset in a single round. We empirically demonstrate the robustness of FeCA under various federated scenarios on both synthetic and real-world data. Additionally, we extend FeCA to representation learning and present DeepFeCA, which combines DeepCluster and FeCA for unsupervised feature learning in the federated setting.