FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Models

TL;DR

FedDEO introduces a novel method for one-shot federated learning that uses locally trained descriptions and diffusion models to generate synthetic data, improving communication efficiency and privacy while achieving superior performance.

Contribution

The paper proposes a new approach that trains local descriptions on clients to guide diffusion models, enabling effective knowledge transfer without public datasets or uniform feature extractors.

Findings

Synthetic datasets generated with high quality and diversity.

Outperforms existing federated learning and diffusion-based OSFL methods.

Achieves performance surpassing centralized training on some clients.

Abstract

In recent years, the attention towards One-Shot Federated Learning (OSFL) has been driven by its capacity to minimize communication. With the development of the diffusion model (DM), several methods employ the DM for OSFL, utilizing model parameters, image features, or textual prompts as mediums to transfer the local client knowledge to the server. However, these mediums often require public datasets or the uniform feature extractor, significantly limiting their practicality. In this paper, we propose FedDEO, a Description-Enhanced One-Shot Federated Learning Method with DMs, offering a novel exploration of utilizing the DM in OSFL. The core idea of our method involves training local descriptions on the clients, serving as the medium to transfer the knowledge of the distributed clients to the server. Firstly, we train local descriptions on the client data to capture the characteristics of client distributions, which are then uploaded to the server. On the server, the descriptions are used as conditions to guide the DM in generating synthetic datasets that comply with the distributions of various clients, enabling the training of the aggregated model. Theoretical analyses and sufficient quantitation and visualization experiments on three large-scale real-world datasets demonstrate that through the training of local descriptions, the server is capable of generating synthetic datasets with high quality and diversity. Consequently, with advantages in communication and privacy protection, the aggregated model outperforms compared FL or diffusion-based OSFL methods and, on some clients, outperforms the performance ceiling of centralized training.