Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies

TL;DR

This paper reviews existing clustered federated learning methods, introduces a comprehensive four-tier framework called HCFL, and proposes an improved clustering method HCFL+ to address persistent challenges, validated through extensive experiments.

Contribution

The paper presents a novel four-tier framework HCFL for clustered FL and an improved clustering method HCFL+ to enhance performance and address existing challenges.

Findings

HCFL framework effectively encompasses existing methods

HCFL+ improves clustering accuracy and robustness

Extensive evaluations validate the proposed methods' effectiveness

Abstract

Federated Learning (FL) is an evolving distributed machine learning approach that safeguards client privacy by keeping data on edge devices. However, the variation in data among clients poses challenges in training models that excel across all local distributions. Recent studies suggest clustering as a solution to address client heterogeneity in FL by grouping clients with distribution shifts into distinct clusters. Nonetheless, the diverse learning frameworks used in current clustered FL methods create difficulties in integrating these methods, leveraging their advantages, and making further enhancements. To this end, this paper conducts a thorough examination of existing clustered FL methods and introduces a four-tier framework, named HCFL, to encompass and extend the existing approaches. Utilizing the HCFL, we identify persistent challenges associated with current clustering methods in each tier and propose an enhanced clustering method called HCFL$^{+}$ to overcome these challenges. Through extensive numerical evaluations, we demonstrate the effectiveness of our clustering framework and the enhanced components. Our code is available at https://github.com/LINs-lab/HCFL.