Modeling Global Distribution for Federated Learning with Label Distribution Skew

TL;DR

This paper introduces FedMGD, a federated learning approach that uses a global GAN to model data distribution across clients with label skew, improving global model performance without privacy compromise.

Contribution

The paper proposes FedMGD, a novel federated learning method utilizing a global GAN to address label distribution skew without accessing local data.

Findings

FedMGD outperforms state-of-the-art methods on public benchmarks.

The global GAN effectively models data distribution without privacy leakage.

Experimental results show significant performance improvements.

Abstract

Federated learning achieves joint training of deep models by connecting decentralized data sources, which can significantly mitigate the risk of privacy leakage. However, in a more general case, the distributions of labels among clients are different, called ``label distribution skew''. Directly applying conventional federated learning without consideration of label distribution skew issue significantly hurts the performance of the global model. To this end, we propose a novel federated learning method, named FedMGD, to alleviate the performance degradation caused by the label distribution skew issue. It introduces a global Generative Adversarial Network to model the global data distribution without access to local datasets, so the global model can be trained using the global information of data distribution without privacy leakage. The experimental results demonstrate that our proposed method significantly outperforms the state-of-the-art on several public benchmarks. Code is available at \url{https://github.com/Sheng-T/FedMGD}.