Hybrid Federated and Centralized Learning

TL;DR

This paper introduces a hybrid learning framework combining federated and centralized learning to enable resource-diverse clients to collaborate effectively, reducing training time and communication overhead while maintaining high accuracy.

Contribution

The paper proposes a novel HFCL framework that allows resource-constrained clients to participate via centralized learning, improving collaboration and efficiency in distributed settings.

Findings

HFCL outperforms non-hybrid schemes in accuracy and communication efficiency.

HFCL-SDT reduces training duration significantly.

All clients contribute to learning regardless of computational resources.

Abstract

Many of the machine learning (ML) tasks are focused on centralized learning (CL), which requires the transmission of local datasets from the clients to a parameter server (PS) leading to a huge communication overhead. Federated learning (FL) overcomes this issue by allowing the clients to send only the model updates to the PS instead of the whole dataset. In this way, FL brings the learning to edge level, wherein powerful computational resources are required on the client side. This requirement may not always be satisfied because of diverse computational capabilities of edge devices. We address this through a novel hybrid federated and centralized learning (HFCL) framework to effectively train a learning model by exploiting the computational capability of the clients. In HFCL, only the clients who have sufficient resources employ FL; the remaining clients resort to CL by transmitting their local dataset to PS. This allows all the clients to collaborate on the learning process regardless of their computational resources. We also propose a sequential data transmission approach with HFCL (HFCL-SDT) to reduce the training duration. The proposed HFCL frameworks outperform previously proposed non-hybrid FL (CL) based schemes in terms of learning accuracy (communication overhead) since all the clients collaborate on the learning process with their datasets regardless of their computational resources.