An Efficient Virtual Data Generation Method for Reducing Communication in Federated Learning

TL;DR

This paper introduces FedINIBoost, a novel data-based framework for reducing communication in federated learning by constructing proxy datasets through gradient matching, enhancing model finetuning efficiency.

Contribution

The paper proposes FedINIBoost, a new method that constructs proxy datasets using gradient matching, improving communication efficiency and initial model performance in federated learning.

Findings

FedINIBoost outperforms FedAVG, FedProx, Moon, and FedFTG in experiments.

The method significantly improves initial stage model performance.

Extensive experiments confirm the effectiveness of the proposed approach.

Abstract

Communication overhead is one of the major challenges in Federated Learning(FL). A few classical schemes assume the server can extract the auxiliary information about training data of the participants from the local models to construct a central dummy dataset. The server uses the dummy dataset to finetune aggregated global model to achieve the target test accuracy in fewer communication rounds. In this paper, we summarize the above solutions into a data-based communication-efficient FL framework. The key of the proposed framework is to design an efficient extraction module(EM) which ensures the dummy dataset has a positive effect on finetuning aggregated global model. Different from the existing methods that use generator to design EM, our proposed method, FedINIBoost borrows the idea of gradient match to construct EM. Specifically, FedINIBoost builds a proxy dataset of the real dataset in two steps for each participant at each communication round. Then the server aggregates all the proxy datasets to form a central dummy dataset, which is used to finetune aggregated global model. Extensive experiments verify the superiority of our method compared with the existing classical method, FedAVG, FedProx, Moon and FedFTG. Moreover, FedINIBoost plays a significant role in finetuning the performance of aggregated global model at the initial stage of FL.