Privacy-preserving Quantification of Non-IID Degree in Federated Learning

TL;DR

This paper introduces a novel cryptographic method to quantitatively measure the non-IID degree of datasets in federated learning, enabling privacy-preserving comparison of data distributions across clients.

Contribution

It proposes the first formal, quantitative definition of non-IID degree in federated learning using FHE-based CDF estimation, enhancing data distribution understanding without privacy compromise.

Findings

Validated on CIFAR-100 non-IID dataset

Effectively estimates non-IID degree while preserving privacy

Enables clients to compare data distributions securely

Abstract

Federated learning (FL) offers a privacy-preserving approach to machine learning for multiple collaborators without sharing raw data. However, the existence of non-independent and non-identically distributed (non-IID) datasets across different clients presents a significant challenge to FL, leading to a sharp drop in accuracy, reduced efficiency, and hindered implementation. To address the non-IID problem, various methods have been proposed, including clustering and personalized FL frameworks. Nevertheless, to date, a formal quantitative definition of the non-IID degree between different clients' datasets is still missing, hindering the clients from comparing and obtaining an overview of their data distributions with other clients. For the first time, this paper proposes a quantitative definition of the non-IID degree in the federated environment by employing the cumulative distribution function (CDF), called Fully Homomorphic Encryption-based Federated Cumulative Distribution Function (FHE-FCDF). This method utilizes cryptographic primitive fully homomorphic encryption to enable clients to estimate the non-IID degree while ensuring privacy preservation. The experiments conducted on the CIFAR-100 non-IID dataset validate the effectiveness of our proposed method.