FedCM: Federated Learning with Client-level Momentum

TL;DR

FedCM introduces a novel federated learning algorithm that incorporates client-level momentum to address partial participation and heterogeneity, improving stability and performance in real-world scenarios.

Contribution

The paper proposes FedCM, a new federated learning method that uses client-level momentum to enhance stability and accuracy under diverse client conditions.

Findings

FedCM outperforms existing methods in various tasks.

FedCM is robust to different client participation levels.

FedCM effectively corrects bias caused by client heterogeneity.

Abstract

Federated Learning is a distributed machine learning approach which enables model training without data sharing. In this paper, we propose a new federated learning algorithm, Federated Averaging with Client-level Momentum (FedCM), to tackle problems of partial participation and client heterogeneity in real-world federated learning applications. FedCM aggregates global gradient information in previous communication rounds and modifies client gradient descent with a momentum-like term, which can effectively correct the bias and improve the stability of local SGD. We provide theoretical analysis to highlight the benefits of FedCM. We also perform extensive empirical studies and demonstrate that FedCM achieves superior performance in various tasks and is robust to different levels of client numbers, participation rate and client heterogeneity.