MimiC: Combating Client Dropouts in Federated Learning by Mimicking Central Updates

TL;DR

MimiC is a novel federated learning algorithm that mitigates client dropout issues by mimicking central updates, ensuring stable convergence and improved model performance in mobile edge networks.

Contribution

This paper introduces MimiC, a new method that modifies client updates to mimic central updates, addressing convergence issues caused by client dropouts in federated learning.

Findings

MimiC achieves stable convergence despite client dropouts.

MimiC outperforms baseline methods in model accuracy.

Theoretical analysis confirms convergence under proper learning rates.

Abstract

Federated learning (FL) is a promising framework for privacy-preserving collaborative learning, where model training tasks are distributed to clients and only the model updates need to be collected at a server. However, when being deployed at mobile edge networks, clients may have unpredictable availability and drop out of the training process, which hinders the convergence of FL. This paper tackles such a critical challenge. Specifically, we first investigate the convergence of the classical FedAvg algorithm with arbitrary client dropouts. We find that with the common choice of a decaying learning rate, FedAvg oscillates around a stationary point of the global loss function, which is caused by the divergence between the aggregated and desired central update. Motivated by this new observation, we then design a novel training algorithm named MimiC, where the server modifies each received model update based on the previous ones. The proposed modification of the received model updates mimics the imaginary central update irrespective of dropout clients. The theoretical analysis of MimiC shows that divergence between the aggregated and central update diminishes with proper learning rates, leading to its convergence. Simulation results further demonstrate that MimiC maintains stable convergence performance and learns better models than the baseline methods.