Theoretical Analysis of Impact of Delayed Updates on Decentralized Federated Learning

TL;DR

This paper provides a theoretical analysis of how delayed parameter updates affect convergence in decentralized federated learning, offering bounds and strategies for maintaining model performance amid network delays.

Contribution

It introduces a theoretical framework and bounds for delayed updates in decentralized federated learning, enhancing understanding of convergence under network delays.

Findings

Theoretical bounds for update delays ensuring convergence.

Reusing latest delayed updates within bounds maintains model accuracy.

Analysis applicable to various edge network topologies.

Abstract

Decentralized Federated learning is a distributed edge intelligence framework by exchanging parameter updates instead of training data among participators, in order to retrain or fine-tune deep learning models for mobile intelligent applications. Considering the various topologies of edge networks in mobile internet, the impact of transmission delay of updates during model training is non-negligible for data-intensive intelligent applications on mobile devices, e.g., intelligent medical services, automated driving vehicles, etc.. To address this problem, we analyze the impact of delayed updates for decentralized federated learning, and provide a theoretical bound for these updates to achieve model convergence. Within the theoretical bound of updating period, the latest versions for the delayed updates are reused to continue aggregation, in case the model parameters from a specific neighbor are not collected or updated in time.