A review on different techniques used to combat the non-IID and heterogeneous nature of data in FL

TL;DR

This paper reviews various techniques to address the challenges posed by non-IID and heterogeneous data distributions in Federated Learning, emphasizing the importance of privacy and model convergence.

Contribution

It provides a comprehensive overview of current algorithms designed to mitigate non-IID and heterogeneity issues in Federated Learning.

Findings

Identifies key challenges of non-IID data in FL

Summarizes existing algorithms addressing data heterogeneity

Highlights gaps and future directions in FL research

Abstract

Federated Learning (FL) is a machine-learning approach enabling collaborative model training across multiple decentralized edge devices that hold local data samples, all without exchanging these samples. This collaborative process occurs under the supervision of a central server orchestrating the training or via a peer-to-peer network. The significance of FL is particularly pronounced in industries such as healthcare and finance, where data privacy holds paramount importance. However, training a model under the Federated learning setting brings forth several challenges, with one of the most prominent being the heterogeneity of data distribution among the edge devices. The data is typically non-independently and non-identically distributed (non-IID), thereby presenting challenges to model convergence. This report delves into the issues arising from non-IID and heterogeneous data and explores current algorithms designed to address these challenges.