DENSE: Data-Free One-Shot Federated Learning

TL;DR

DENSE introduces a practical one-shot federated learning framework that trains models without additional data or information, accommodating heterogeneous client models and outperforming existing methods.

Contribution

DENSE is the first data-free, one-shot federated learning method that handles model heterogeneity and does not require auxiliary datasets or extra information.

Findings

DENSE outperforms Fed-ADI by 5.08% on CIFAR10.

The method works effectively across various real-world datasets.

DENSE requires only model parameters for communication, with no auxiliary data.

Abstract

One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round. Despite the low communication cost, existing one-shot FL methods are mostly impractical or face inherent limitations, \eg a public dataset is required, clients' models are homogeneous, and additional data/model information need to be uploaded. To overcome these issues, we propose a novel two-stage \textbf{D}ata-fre\textbf{E} o\textbf{N}e-\textbf{S}hot federated l\textbf{E}arning (DENSE) framework, which trains the global model by a data generation stage and a model distillation stage. DENSE is a practical one-shot FL method that can be applied in reality due to the following advantages: (1) DENSE requires no additional information compared with other methods (except the model parameters) to be transferred between clients and the server; (2) DENSE does not require any auxiliary dataset for training; (3) DENSE considers model heterogeneity in FL, \ie different clients can have different model architectures. Experiments on a variety of real-world datasets demonstrate the superiority of our method.For example, DENSE outperforms the best baseline method Fed-ADI by 5.08\% on CIFAR10 dataset.