Communication-Efficient Federated Learning with Adaptive Number of Participants

TL;DR

This paper proposes an adaptive participant selection mechanism in federated learning to improve communication efficiency, demonstrating up to 30% savings without accuracy loss across various applications.

Contribution

It introduces ISP, a novel adaptive method for dynamically choosing the number of clients per round in federated learning, addressing a largely unexplored problem.

Findings

Achieves up to 30% communication savings

Maintains model accuracy with fewer clients

Effective across diverse real-world applications

Abstract

Rapid scaling of deep learning models has enabled performance gains across domains, yet it introduced several challenges. Federated Learning (FL) has emerged as a promising framework to address these concerns by enabling decentralized training. Nevertheless, communication efficiency remains a key bottleneck in FL, particularly under heterogeneous and dynamic client participation. Existing methods, such as FedAvg and FedProx, or other approaches, including client selection strategies, attempt to mitigate communication costs. However, the problem of choosing the number of clients in a training round remains extremely underexplored. We introduce Intelligent Selection of Participants (ISP), an adaptive mechanism that dynamically determines the optimal number of clients per round to enhance communication efficiency without compromising model accuracy. We validate the effectiveness of ISP across diverse setups, including vision transformers, real-world ECG classification, and training with gradient compression. Our results show consistent communication savings of up to 30\% without losing the final quality. Applying ISP to different real-world ECG classification setups highlighted the selection of the number of clients as a separate task of federated learning.