FedADC: Accelerated Federated Learning with Drift Control

TL;DR

FedADC is a novel federated learning algorithm that accelerates training and controls data drift, effectively addressing two major challenges in large-scale distributed FL without extra computational or communication costs.

Contribution

The paper introduces FedADC, a unified approach that combines acceleration and drift control in federated learning, improving efficiency and robustness.

Findings

FedADC outperforms existing FL methods in convergence speed.

It effectively mitigates data drift in non-i.i.d. settings.

Empirical results demonstrate enhanced training efficiency.

Abstract

Federated learning (FL) has become de facto framework for collaborative learning among edge devices with privacy concern. The core of the FL strategy is the use of stochastic gradient descent (SGD) in a distributed manner. Large scale implementation of FL brings new challenges, such as the incorporation of acceleration techniques designed for SGD into the distributed setting, and mitigation of the drift problem due to non-homogeneous distribution of local datasets. These two problems have been separately studied in the literature; whereas, in this paper, we show that it is possible to address both problems using a single strategy without any major alteration to the FL framework, or introducing additional computation and communication load. To achieve this goal, we propose FedADC, which is an accelerated FL algorithm with drift control. We empirically illustrate the advantages of FedADC.