Resource Aware Clustering for Tackling the Heterogeneity of Participants in Federated Learning

TL;DR

This paper introduces a resource-aware clustering method for federated learning that addresses device heterogeneity, optimizing training efficiency through clustering, participant assignment, and knowledge distillation, validated by experimental results.

Contribution

It proposes a novel resource-aware clustering approach with optimal cluster determination and a master-slave technique to enhance federated learning performance.

Findings

Improved training efficiency in heterogeneous environments

Effective clustering based on resource information

Enhanced lightweight model performance via knowledge distillation

Abstract

Federated Learning is a training framework that enables multiple participants to collaboratively train a shared model while preserving data privacy and minimizing communication overhead. The heterogeneity of devices and networking resources of the participants delay the training and aggregation in federated learning. This paper proposes a federated learning approach to manoeuvre the heterogeneity among the participants using resource aware clustering. The approach begins with the server gathering information about the devices and networking resources of participants, after which resource aware clustering is performed to determine the optimal number of clusters using Dunn Indices. The mechanism of participant assignment is then introduced, and the expression of communication rounds required for model convergence in each cluster is mathematically derived. Furthermore, a master-slave technique is introduced to improve the performance of the lightweight models in the clusters using knowledge distillation. Finally, experimental evaluations are conducted to verify the feasibility and effectiveness of the approach and to compare it with state-of-the-art techniques.