Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets

TL;DR

This paper introduces Fed-CHS, a novel federated learning algorithm that combines hierarchical and sequential approaches to reduce communication overhead and improve accuracy in non-IID data settings.

Contribution

It proposes a new sequential federated learning framework within hierarchical architecture and derives convergence guarantees for various loss functions.

Findings

Fed-CHS reduces communication overhead significantly.

Fed-CHS achieves comparable or better test accuracy.

Experimental results validate the efficiency of Fed-CHS.

Abstract

In a real federated learning (FL) system, communication overhead for passing model parameters between the clients and the parameter server (PS) is often a bottleneck. Hierarchical federated learning (HFL) that poses multiple edge servers (ESs) between clients and the PS can partially alleviate communication pressure but still needs the aggregation of model parameters from multiple ESs at the PS. To further reduce communication overhead, we bring sequential FL (SFL) into HFL for the first time, which removes the central PS and enables the model training to be completed only through passing the global model between two adjacent ESs for each iteration, and propose a novel algorithm adaptive to such a combinational framework, referred to as Fed-CHS. Convergence results are derived for strongly convex and non-convex loss functions under various data heterogeneity setups, which show comparable convergence performance with the algorithms for HFL or SFL solely. Experimental results provide evidence of the superiority of our proposed Fed-CHS on both communication overhead saving and test accuracy over baseline methods.