Aggregation Delayed Federated Learning

TL;DR

This paper introduces a novel federated learning framework with delayed aggregation rounds, called aggregation delayed federated learning, which improves performance on non-IID data by addressing dataset heterogeneity.

Contribution

It proposes a new approach of delaying aggregation rounds in federated learning to better handle non-IID data distributions, differing from existing methods.

Findings

Significant performance improvements on non-IID datasets.

Effective handling of data heterogeneity in federated learning.

Outperforms standard FedAvg in experiments.

Abstract

Federated learning is a distributed machine learning paradigm where multiple data owners (clients) collaboratively train one machine learning model while keeping data on their own devices. The heterogeneity of client datasets is one of the most important challenges of federated learning algorithms. Studies have found performance reduction with standard federated algorithms, such as FedAvg, on non-IID data. Many existing works on handling non-IID data adopt the same aggregation framework as FedAvg and focus on improving model updates either on the server side or on clients. In this work, we tackle this challenge in a different view by introducing redistribution rounds that delay the aggregation. We perform experiments on multiple tasks and show that the proposed framework significantly improves the performance on non-IID data.