FedSR: A Semi-Decentralized Federated Learning Algorithm for Non-IIDness in IoT System

TL;DR

This paper introduces FedSR, a semi-decentralized federated learning framework for IoT that mitigates data heterogeneity and reduces communication bottlenecks by combining centralized and decentralized approaches.

Contribution

It proposes a novel hierarchical federated learning framework with an incremental subgradient algorithm to improve model generalization across heterogeneous IoT devices.

Findings

Effectively mitigates data heterogeneity impact

Reduces communication bottleneck in cloud servers

Enhances scalability for large IoT deployments

Abstract

In the Industrial Internet of Things (IoT), a large amount of data will be generated every day. Due to privacy and security issues, it is difficult to collect all these data together to train deep learning models, thus the federated learning, a distributed machine learning paradigm that protects data privacy, has been widely used in IoT. However, in practical federated learning, the data distributions usually have large differences across devices, and the heterogeneity of data will deteriorate the performance of the model. Moreover, federated learning in IoT usually has a large number of devices involved in training, and the limited communication resource of cloud servers become a bottleneck for training. To address the above issues, in this paper, we combine centralized federated learning with decentralized federated learning to design a semi-decentralized cloud-edge-device hierarchical federated learning framework, which can mitigate the impact of data heterogeneity, and can be deployed at lage scale in IoT. To address the effect of data heterogeneity, we use an incremental subgradient optimization algorithm in each ring cluster to improve the generalization ability of the ring cluster models. Our extensive experiments show that our approach can effectively mitigate the impact of data heterogeneity and alleviate the communication bottleneck in cloud servers.