SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning

TL;DR

SemiDFL introduces a novel semi-supervised decentralized federated learning approach that leverages neighborhood information, consensus-based data synthesis, and adaptive aggregation to improve performance with unlabeled data across clients.

Contribution

It is the first semi-supervised DFL method that effectively combines pseudo-labeling, data synthesis, and adaptive aggregation to handle unlabeled data in decentralized settings.

Findings

SemiDFL outperforms existing CFL and DFL methods in various SSL scenarios.

The method effectively leverages unlabeled data through pseudo-labeling and data synthesis.

SemiDFL achieves superior accuracy in both IID and non-IID data distributions.

Abstract

Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semisupervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabeled data. We then design a consensusbased diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both IID and non-IID SSL scenarios.