Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learning

TL;DR

This paper introduces AFIND+, an efficient neighbor sampling and aggregation algorithm for decentralized federated learning, which improves model performance by selectively collaborating with helpful neighbors.

Contribution

The paper proposes AFIND+, a novel algorithm that adaptively samples and aggregates neighbors in DFL to enhance local model accuracy, addressing data heterogeneity issues.

Findings

AFIND+ outperforms existing sampling algorithms in DFL.

AFIND+ is compatible with most DFL optimization methods.

Numerical results show improved model performance on real-world datasets.

Abstract

Federated Learning (FL) is gaining widespread interest for its ability to share knowledge while preserving privacy and reducing communication costs. Unlike Centralized FL, Decentralized FL (DFL) employs a network architecture that eliminates the need for a central server, allowing direct communication among clients and leading to significant communication resource savings. However, due to data heterogeneity, not all neighboring nodes contribute to enhancing the local client's model performance. In this work, we introduce \textbf{\emph{AFIND+}}, a simple yet efficient algorithm for sampling and aggregating neighbors in DFL, with the aim of leveraging collaboration to improve clients' model performance. AFIND+ identifies helpful neighbors, adaptively adjusts the number of selected neighbors, and strategically aggregates the sampled neighbors' models based on their contributions. Numerical results on real-world datasets with diverse data partitions demonstrate that AFIND+ outperforms other sampling algorithms in DFL and is compatible with most existing DFL optimization algorithms.