Recovering Global Data Distribution Locally in Federated Learning

TL;DR

This paper introduces ReGL, a client-side generative approach in federated learning that synthesizes data to address label imbalance, improving model performance without compromising data privacy.

Contribution

ReGL is the first method to recover global data distribution locally in FL using generative models, effectively alleviating label imbalance without data sharing.

Findings

ReGL outperforms existing methods in handling label imbalance.

Synthetic data improves local model accuracy.

Client-side generation preserves privacy.

Abstract

Federated Learning (FL) is a distributed machine learning paradigm that enables collaboration among multiple clients to train a shared model without sharing raw data. However, a major challenge in FL is the label imbalance, where clients may exclusively possess certain classes while having numerous minority and missing classes. Previous works focus on optimizing local updates or global aggregation but ignore the underlying imbalanced label distribution across clients. In this paper, we propose a novel approach ReGL to address this challenge, whose key idea is to Recover the Global data distribution Locally. Specifically, each client uses generative models to synthesize images that complement the minority and missing classes, thereby alleviating label imbalance. Moreover, we adaptively fine-tune the image generation process using local real data, which makes the synthetic images align more closely with the global distribution. Importantly, both the generation and fine-tuning processes are conducted at the client-side without leaking data privacy. Through comprehensive experiments on various image classification datasets, we demonstrate the remarkable superiority of our approach over existing state-of-the-art works in fundamentally tackling label imbalance in FL.