Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach

TL;DR

This paper introduces a contribution-aware asynchronous federated learning method that dynamically adjusts update contributions based on staleness and data heterogeneity, improving convergence speed in realistic settings.

Contribution

It presents a novel asynchronous FL algorithm that accounts for update contribution factors, enhancing convergence over existing methods.

Findings

Faster convergence compared to traditional asynchronous methods.

Effective handling of staleness and heterogeneity in updates.

Improved training efficiency in federated learning scenarios.

Abstract

Federated Learning (FL) is a distributed machine learning paradigm that allows clients to train models on their data while preserving their privacy. FL algorithms, such as Federated Averaging (FedAvg) and its variants, have been shown to converge well in many scenarios. However, these methods require clients to upload their local updates to the server in a synchronous manner, which can be slow and unreliable in realistic FL settings. To address this issue, researchers have developed asynchronous FL methods that allow clients to continue training on their local data using a stale global model. However, most of these methods simply aggregate all of the received updates without considering their relative contributions, which can slow down convergence. In this paper, we propose a contribution-aware asynchronous FL method that takes into account the staleness and statistical heterogeneity of the received updates. Our method dynamically adjusts the contribution of each update based on these factors, which can speed up convergence compared to existing methods.