Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks

TL;DR

This paper introduces HierMo, a hierarchical federated learning algorithm with momentum acceleration across multi-tier networks, providing convergence guarantees and optimized aggregation strategies for faster training.

Contribution

It presents HierMo, a novel three-tier federated learning method with momentum acceleration and convergence analysis, along with HierOPT for optimizing aggregation periods.

Findings

HierMo achieves a convergence rate of O(1/T).

HierMo has a tighter convergence upper bound than HierFAVG without momentum.

HierOPT effectively minimizes loss by optimizing aggregation periods.

Abstract

In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, {we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum}. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time.