Personalized Federated Learning with Local Attention

TL;DR

This paper introduces pFedLA, a personalized federated learning method that uses local attention mechanisms to effectively address feature shift heterogeneity across clients, improving performance without extra communication costs.

Contribution

The paper proposes a novel local attention-based approach for personalized federated learning that handles feature shift heterogeneity and can be integrated with existing FL methods.

Findings

pFedLA improves performance on image classification tasks.

The method effectively addresses feature shift heterogeneity.

It can be incorporated into various FL algorithms without additional communication overhead.

Abstract

Federated Learning (FL) aims to learn a single global model that enables the central server to help the model training in local clients without accessing their local data. The key challenge of FL is the heterogeneity of local data in different clients, such as heterogeneous label distribution and feature shift, which could lead to significant performance degradation of the learned models. Although many studies have been proposed to address the heterogeneous label distribution problem, few studies attempt to explore the feature shift issue. To address this issue, we propose a simple yet effective algorithm, namely \textbf{p}ersonalized \textbf{Fed}erated learning with \textbf{L}ocal \textbf{A}ttention (pFedLA), by incorporating the attention mechanism into personalized models of clients while keeping the attention blocks client-specific. Specifically, two modules are proposed in pFedLA, i.e., the personalized single attention module and the personalized hybrid attention module. In addition, the proposed pFedLA method is quite flexible and general as it can be incorporated into any FL method to improve their performance without introducing additional communication costs. Extensive experiments demonstrate that the proposed pFedLA method can boost the performance of state-of-the-art FL methods on different tasks such as image classification and object detection tasks.