Accelerated Gradient Descent Learning over Multiple Access Fading Channels

TL;DR

This paper introduces AGMA, a novel accelerated gradient descent algorithm for distributed learning over wireless channels, which improves convergence rates without complex power control or beamforming.

Contribution

The paper develops AGMA, a momentum-based distributed learning algorithm that handles noisy fading channels and achieves faster convergence without requiring power control.

Findings

AGMA approaches the best linear convergence rate for strongly convex functions.

AGMA significantly improves sub-linear convergence for convex functions.

Simulation results demonstrate better performance of AGMA on real datasets.

Abstract

We consider a distributed learning problem in a wireless network, consisting of N distributed edge devices and a parameter server (PS). The objective function is a sum of the edge devices' local loss functions, who aim to train a shared model by communicating with the PS over multiple access channels (MAC). This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning, known as over-the-air computation. In this paper, we develop a novel Accelerated Gradient-descent Multiple Access (AGMA) algorithm that uses momentum-based gradient signals over noisy fading MAC to improve the convergence rate as compared to existing methods. Furthermore, AGMA does not require power control or beamforming to cancel the fading effect, which simplifies the implementation complexity. We analyze AGMA theoretically, and establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. For the strongly convex case, we show that AGMA approaches the best-known linear convergence rate as the network increases. For the convex case, we show that AGMA significantly improves the sub-linear convergence rate as compared to existing methods. Finally, we present simulation results using real datasets that demonstrate better performance by AGMA.