AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

TL;DR

This paper introduces AEDFL, an asynchronous decentralized federated learning framework that improves efficiency, accuracy, and reduces communication costs in heterogeneous device environments through novel aggregation, staleness-awareness, and sparse training techniques.

Contribution

The paper presents a novel asynchronous decentralized FL framework with an efficient aggregation, staleness-aware updates, and adaptive sparse training, addressing key limitations of existing methods.

Findings

Achieves up to 16.3% higher accuracy

Up to 92.9% faster training efficiency

Reduces computation costs by up to 42.3%

Abstract

Federated Learning (FL) has achieved significant achievements recently, enabling collaborative model training on distributed data over edge devices. Iterative gradient or model exchanges between devices and the centralized server in the standard FL paradigm suffer from severe efficiency bottlenecks on the server. While enabling collaborative training without a central server, existing decentralized FL approaches either focus on the synchronous mechanism that deteriorates FL convergence or ignore device staleness with an asynchronous mechanism, resulting in inferior FL accuracy. In this paper, we propose an Asynchronous Efficient Decentralized FL framework, i.e., AEDFL, in heterogeneous environments with three unique contributions. First, we propose an asynchronous FL system model with an efficient model aggregation method for improving the FL convergence. Second, we propose a dynamic staleness-aware model update approach to achieve superior accuracy. Third, we propose an adaptive sparse training method to reduce communication and computation costs without significant accuracy degradation. Extensive experimentation on four public datasets and four models demonstrates the strength of AEDFL in terms of accuracy (up to 16.3% higher), efficiency (up to 92.9% faster), and computation costs (up to 42.3% lower).