Asynchronous Hierarchical Federated Learning

TL;DR

This paper introduces an asynchronous hierarchical federated learning framework that reduces server load, improves convergence speed, and handles system heterogeneity, demonstrated through CIFAR-10 image classification experiments.

Contribution

It proposes a novel asynchronous hierarchical federated learning approach with clustering and aggregation strategies to enhance efficiency and robustness.

Findings

Reduced server communication load.

Faster convergence compared to traditional methods.

Effective handling of system heterogeneity.

Abstract

Federated Learning is a rapidly growing area of research and with various benefits and industry applications. Typical federated patterns have some intrinsic issues such as heavy server traffic, long periods of convergence, and unreliable accuracy. In this paper, we address these issues by proposing asynchronous hierarchical federated learning, in which the central server uses either the network topology or some clustering algorithm to assign clusters for workers (i.e., client devices). In each cluster, a special aggregator device is selected to enable hierarchical learning, leads to efficient communication between server and workers, so that the burden of the server can be significantly reduced. In addition, asynchronous federated learning schema is used to tolerate heterogeneity of the system and achieve fast convergence, i.e., the server aggregates the gradients from the workers weighted by a staleness parameter to update the global model, and regularized stochastic gradient descent is performed in workers, so that the instability of asynchronous learning can be alleviated. We evaluate the proposed algorithm on CIFAR-10 image classification task, the experimental results demonstrate the effectiveness of asynchronous hierarchical federated learning.