Towards Fast and Accurate Federated Learning with non-IID Data for Cloud-Based IoT Applications

TL;DR

This paper introduces a novel device grouping method based on feature map similarity using Locality-Sensitive Hashing to enhance federated learning with non-IID data in IoT, achieving faster convergence and higher accuracy while preserving privacy.

Contribution

A new feature map similarity-based device grouping technique using LSH that reduces divergence and improves federated learning performance on non-IID IoT data.

Findings

Accelerates convergence rate of federated learning.

Improves prediction accuracy with non-IID data.

Maintains privacy by using LSH for similarity computation.

Abstract

As a promising method of central model training on decentralized device data while securing user privacy, Federated Learning (FL)is becoming popular in Internet of Things (IoT) design. However, when the data collected by IoT devices are highly skewed in a non-independent and identically distributed (non-IID) manner, the accuracy of vanilla FL method cannot be guaranteed. Although there exist various solutions that try to address the bottleneck of FL with non-IID data, most of them suffer from extra intolerable communication overhead and low model accuracy. To enable fast and accurate FL, this paper proposes a novel data-based device grouping approach that can effectively reduce the disadvantages of weight divergence during the training of non-IID data. However, since our grouping method is based on the similarity of extracted feature maps from IoT devices, it may incur additional risks of privacy exposure. To solve this problem, we propose an improved version by exploiting similarity information using the Locality-Sensitive Hashing (LSH) algorithm without exposing extracted feature maps. Comprehensive experimental results on well-known benchmarks show that our approach can not only accelerate the convergence rate, but also improve the prediction accuracy for FL with non-IID data.