CSMAAFL: Client Scheduling and Model Aggregation in Asynchronous Federated Learning

TL;DR

This paper proposes a novel asynchronous federated learning framework with client scheduling and model aggregation strategies that match synchronous convergence speed while accelerating early training.

Contribution

It introduces effective aggregation solutions and client scheduling methods to improve convergence and training speed in asynchronous federated learning environments.

Findings

Achieves similar accuracy to synchronous FL

Accelerates early-stage training

Improves convergence with new aggregation strategies

Abstract

Asynchronous federated learning aims to solve the straggler problem in heterogeneous environments, i.e., clients have small computational capacities that could cause aggregation delay. The principle of asynchronous federated learning is to allow the server to aggregate the model once it receives an update from any client rather than waiting for updates from multiple clients or waiting a specified amount of time in the synchronous mode. Due to the asynchronous setting, the stale model problem could occur, where the slow clients could utilize an outdated local model for their local data training. Consequently, when these locally trained models are uploaded to the server, they may impede the convergence of the global training. Therefore, effective model aggregation strategies play a significant role in updating the global model. Besides, client scheduling is also critical when heterogeneous clients with diversified computing capacities are participating in the federated learning process. This work first investigates the impact of the convergence of asynchronous federated learning mode when adopting the aggregation coefficient in synchronous mode. The effective aggregation solutions that can achieve the same convergence result as in the synchronous mode are then proposed, followed by an improved aggregation method with client scheduling. The simulation results in various scenarios demonstrate that the proposed algorithm converges with a similar level of accuracy as the classical synchronous federated learning algorithm but effectively accelerates the learning process, especially in its early stage.