Fast-Convergent Federated Learning via Cyclic Aggregation

TL;DR

This paper introduces a cyclic aggregation method in federated learning that significantly reduces training iterations and enhances performance without extra computational costs, addressing convergence issues in heterogeneous environments.

Contribution

It proposes a cyclic learning rate scheme at the server to accelerate convergence and improve performance in federated learning without additional computational overhead.

Findings

Reduces training iterations in federated learning.

Improves model performance in heterogeneous settings.

Validated through numerical experiments.

Abstract

Federated learning (FL) aims at optimizing a shared global model over multiple edge devices without transmitting (private) data to the central server. While it is theoretically well-known that FL yields an optimal model -- centrally trained model assuming availability of all the edge device data at the central server -- under mild condition, in practice, it often requires massive amount of iterations until convergence, especially under presence of statistical/computational heterogeneity. This paper utilizes cyclic learning rate at the server side to reduce the number of training iterations with increased performance without any additional computational costs for both the server and the edge devices. Numerical results validate that, simply plugging-in the proposed cyclic aggregation to the existing FL algorithms effectively reduces the number of training iterations with improved performance.