Scalable Hierarchical Over-the-Air Federated Learning

TL;DR

This paper proposes a scalable hierarchical federated learning framework over wireless networks that effectively manages interference and data heterogeneity, ensuring high accuracy and outperforming traditional methods.

Contribution

It introduces a novel two-level learning method with over-the-air aggregation and bandwidth-efficient broadcast schemes, along with a comprehensive convergence analysis under interference.

Findings

Achieves high learning accuracy despite interference and heterogeneity.

Outperforms conventional hierarchical learning algorithms.

Provides a mathematical convergence bound for multi-cluster wireless networks.

Abstract

When implementing hierarchical federated learning over wireless networks, scalability assurance and the ability to handle both interference and device data heterogeneity are crucial. This work introduces a new two-level learning method designed to address these challenges, along with a scalable over-the-air aggregation scheme for the uplink and a bandwidth-limited broadcast scheme for the downlink that efficiently use a single wireless resource. To provide resistance against data heterogeneity, we employ gradient aggregations. Meanwhile, the impact of uplink and downlink interference is minimized through optimized receiver normalizing factors. We present a comprehensive mathematical approach to derive the convergence bound for the proposed algorithm, applicable to a multi-cluster wireless network encompassing any count of collaborating clusters, and provide special cases and design remarks. As a key step to enable a tractable analysis, we develop a spatial model for the setup by modeling devices as a Poisson cluster process over the edge servers and rigorously quantify uplink and downlink error terms due to the interference. Finally, we show that despite the interference and data heterogeneity, the proposed algorithm not only achieves high learning accuracy for a variety of parameters but also significantly outperforms the conventional hierarchical learning algorithm.