Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping

TL;DR

This paper introduces Median Anchored Clipping (MAC), a new method to improve federated learning over noisy radio channels by reducing the impact of heavy-tailed electromagnetic interference, thus enhancing robustness and convergence.

Contribution

The paper proposes MAC, a novel gradient clipping technique specifically designed for over-the-air federated learning to combat heavy-tailed noise, with analytical convergence analysis and extensive experiments.

Findings

MAC effectively mitigates heavy-tailed noise impact.

Improves convergence rate and training robustness.

Enhances system privacy and reduces communication costs.

Abstract

Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness.