FedABC: Attention-Based Client Selection for Federated Learning with Long-Term View

TL;DR

FedABC introduces an attention-based client selection method for federated learning that improves model accuracy and efficiency by considering long-term participation and data heterogeneity, outperforming existing methods.

Contribution

The paper presents FedABC, a novel client selection algorithm using attention mechanisms and a long-term optimization strategy to enhance federated learning performance.

Findings

FedABC achieves 3.5% higher accuracy than state-of-the-art methods.

FedABC reduces client participation needs by 32% compared to FedAvg.

The approach improves model accuracy and efficiency in resource-constrained environments.

Abstract

Native AI support is a key objective in the evolution of 6G networks, with Federated Learning (FL) emerging as a promising paradigm. FL allows decentralized clients to collaboratively train an AI model without directly sharing their data, preserving privacy. Clients train local models on private data and share model updates, which a central server aggregates to refine the global model and redistribute it for the next iteration. However, client data heterogeneity slows convergence and reduces model accuracy, and frequent client participation imposes communication and computational burdens. To address these challenges, we propose FedABC, an innovative client selection algorithm designed to take a long-term view in managing data heterogeneity and optimizing client participation. Inspired by attention mechanisms, FedABC prioritizes informative clients by evaluating both model similarity and each model's unique contributions to the global model. Moreover, considering the evolving demands of the global model, we formulate an optimization problem to guide FedABC throughout the training process. Following the "later-is-better" principle, FedABC adaptively adjusts the client selection threshold, encouraging greater participation in later training stages. Extensive simulations on CIFAR-10 demonstrate that FedABC significantly outperforms existing approaches in model accuracy and client participation efficiency, achieving comparable performance with 32% fewer clients than the classical FL algorithm FedAvg, and 3.5% higher accuracy with 2% fewer clients than the state-of-the-art. This work marks a step toward deploying FL in heterogeneous, resource-constrained environments, thereby supporting native AI capabilities in 6G networks.