Asynchronous Decentralized Learning over Unreliable Wireless Networks

TL;DR

This paper introduces an asynchronous decentralized stochastic gradient descent algorithm designed for unreliable wireless networks, providing theoretical convergence guarantees and demonstrating its robustness and efficiency through experiments.

Contribution

It presents a novel asynchronous DSGD algorithm that handles failures in wireless networks, with theoretical analysis and empirical validation of its effectiveness.

Findings

Algorithm is robust to communication failures.

Asynchronous updates improve convergence speed.

Outdated gradient reuse benefits decentralized learning.

Abstract

Decentralized learning enables edge users to collaboratively train models by exchanging information via device-to-device communication, yet prior works have been limited to wireless networks with fixed topologies and reliable workers. In this work, we propose an asynchronous decentralized stochastic gradient descent (DSGD) algorithm, which is robust to the inherent computation and communication failures occurring at the wireless network edge. We theoretically analyze its performance and establish a non-asymptotic convergence guarantee. Experimental results corroborate our analysis, demonstrating the benefits of asynchronicity and outdated gradient information reuse in decentralized learning over unreliable wireless networks.