One-Shot Federated Clustering of Non-Independent Completely Distributed Data

TL;DR

This paper introduces GOLD, a novel federated clustering framework designed to handle Non-Independent Completely Distributed data, effectively addressing Non-IID challenges and fragmentations in distributed clustering tasks.

Contribution

GOLD is the first framework to explore incomplete local cluster distributions and perform global fusion for Non-ICD data in federated clustering, improving performance over existing methods.

Findings

GOLD outperforms existing federated clustering methods in Non-ICD scenarios.

The framework effectively mitigates cluster fragmentation caused by Non-IID data.

Extensive experiments validate the scalability and robustness of GOLD.

Abstract

Federated Learning (FL) that extracts data knowledge while protecting the privacy of multiple clients has achieved remarkable results in distributed privacy-preserving IoT systems, including smart traffic flow monitoring, smart grid load balancing, and so on. Since most data collected from edge devices are unlabeled, unsupervised Federated Clustering (FC) is becoming increasingly popular for exploring pattern knowledge from complex distributed data. However, due to the lack of label guidance, the common Non-Independent and Identically Distributed (Non-IID) issue of clients have greatly challenged FC by posing the following problems: How to fuse pattern knowledge (i.e., cluster distribution) from Non-IID clients; How are the cluster distributions among clients related; and How does this relationship connect with the global knowledge fusion? In this paper, a more tricky but overlooked phenomenon in Non-IID is revealed, which bottlenecks the clustering performance of the existing FC approaches. That is, different clients could fragment a cluster, and accordingly, a more generalized Non-IID concept, i.e., Non-ICD (Non-Independent Completely Distributed), is derived. To tackle the above FC challenges, a new framework named GOLD (Global Oriented Local Distribution Learning) is proposed. GOLD first finely explores the potential incomplete local cluster distributions of clients, then uploads the distribution summarization to the server for global fusion, and finally performs local cluster enhancement under the guidance of the global distribution. Extensive experiments, including significance tests, ablation studies, scalability evaluations, qualitative results, etc., have been conducted to show the superiority of GOLD.