Personalized Federated Learning with Attention-based Client Selection

TL;DR

This paper introduces FedACS, a personalized federated learning algorithm that uses attention-based client selection to improve model performance with non-IID data and limited training data, validated through experiments.

Contribution

The paper proposes FedACS, an innovative PFL method incorporating attention-based client selection to enhance collaboration among similar data clients and address data scarcity issues.

Findings

FedACS outperforms existing methods on CIFAR10 and FMNIST.

Attention-based client selection improves model personalization.

Theoretical convergence of FedACS is established.

Abstract

Personalized Federated Learning (PFL) relies on collective data knowledge to build customized models. However, non-IID data between clients poses significant challenges, as collaborating with clients who have diverse data distributions can harm local model performance, especially with limited training data. To address this issue, we propose FedACS, a new PFL algorithm with an Attention-based Client Selection mechanism. FedACS integrates an attention mechanism to enhance collaboration among clients with similar data distributions and mitigate the data scarcity issue. It prioritizes and allocates resources based on data similarity. We further establish the theoretical convergence behavior of FedACS. Experiments on CIFAR10 and FMNIST validate FedACS's superiority, showcasing its potential to advance personalized federated learning. By tackling non-IID data challenges and data scarcity, FedACS offers promising advances in the field of personalized federated learning.