Personalized Cross-Silo Federated Learning on Non-IID Data

TL;DR

This paper introduces FedAMP, a federated learning method that enhances collaboration among similar clients with non-IID data using attentive message passing, improving performance on benchmark datasets.

Contribution

The paper proposes FedAMP, a novel federated learning approach employing attentive message passing for better client collaboration on non-IID data, with proven convergence and performance improvements.

Findings

FedAMP outperforms existing methods on benchmark datasets.

Convergence is established for both convex and non-convex models.

Heuristic improvements enhance deep neural network personalization.

Abstract

Non-IID data present a tough challenge for federated learning. In this paper, we explore a novel idea of facilitating pairwise collaborations between clients with similar data. We propose FedAMP, a new method employing federated attentive message passing to facilitate similar clients to collaborate more. We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Our extensive experiments on benchmark data sets demonstrate the superior performance of the proposed methods.