Straggler Mitigation and Latency Optimization in Blockchain-based Hierarchical Federated Learning

TL;DR

This paper introduces a blockchain-based hierarchical federated learning framework that addresses straggler issues and latency challenges, ensuring reliable and efficient distributed learning in a decentralized environment.

Contribution

It proposes a novel decentralized, straggler-tolerant BHFL framework with a blockchain-based aggregation method and latency optimization strategies, improving robustness and efficiency.

Findings

BHFL converges despite stragglers and non-IID data.

HieAvg outperforms traditional aggregation methods.

Latency is optimized considering convergence and blockchain delays.

Abstract

Cloud-edge-device hierarchical federated learning (HFL) has been recently proposed to achieve communication-efficient and privacy-preserving distributed learning. However, there exist several critical challenges, such as the single point of failure and potential stragglers in both edge servers and local devices. To resolve these issues, we propose a decentralized and straggler-tolerant blockchain-based HFL (BHFL) framework. Specifically, a Raft-based consortium blockchain is deployed on edge servers to provide a distributed and trusted computing environment for global model aggregation in BHFL. To mitigate the influence of stragglers on learning, we propose a novel aggregation method, HieAvg, which utilizes the historical weights of stragglers to estimate the missing submissions. Furthermore, we optimize the overall latency of BHFL by jointly considering the constraints of global model convergence and blockchain consensus delay. Theoretical analysis and experimental evaluation show that our proposed BHFL based on HieAvg can converge in the presence of stragglers, which performs better than the traditional methods even when the loss function is non-convex and the data on local devices are non-independent and identically distributed (non-IID).