Hierarchical Federated Learning with SignSGD: A Highly Communication-Efficient Approach

TL;DR

This paper introduces HierSignSGD, a highly communication-efficient hierarchical federated learning framework using sign-based gradient compression, which maintains accuracy while significantly reducing communication costs in large-scale IoT systems.

Contribution

It develops a novel hierarchical federated learning algorithm with sign-based compression, providing convergence analysis and demonstrating robustness and efficiency in communication-constrained environments.

Findings

Achieves comparable or better accuracy than full-precision SGD.

Reduces communication cost significantly through extreme compression.

Remains robust under aggressive downlink sparsification.

Abstract

Hierarchical federated learning (HFL) has emerged as a key architecture for large-scale wireless and Internet of Things systems, where devices communicate with nearby edge servers before reaching the cloud. In these environments, uplink bandwidth and latency impose strict communication limits, thereby making aggressive gradient compression essential. One-bit methods such as sign-based stochastic gradient descent (SignSGD) offer an attractive solution in flat federated settings, but existing theory and algorithms do not naturally extend to hierarchical settings. In particular, the interaction between majority-vote aggregation at the edge layer and model aggregation at the cloud layer, and its impact on end-to-end performance, remains unknown. To bridge this gap, we propose a highly communication-efficient sign-based HFL framework and develop its corresponding formulation for nonconvex learning, where devices send only signed stochastic gradients, edge servers combine them through majority-vote, and the cloud periodically averages the obtained edge models, while utilizing downlink quantization to broadcast the global model. We introduce the resulting scalable HFL algorithm, HierSignSGD, and provide the convergence analysis for SignSGD in a hierarchical setting. Our core technical contribution is a characterization of how biased sign compression, two-level aggregation intervals, and inter-cluster heterogeneity collectively affect convergence. Numerical experiments under homogeneous and heterogeneous data splits show that HierSignSGD, despite employing extreme compression, achieves accuracy comparable to or better than full-precision stochastic gradient descent while reducing communication cost in the process, and remains robust under aggressive downlink sparsification.