SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling

TL;DR

SemiFed introduces a federated learning framework that leverages semi-supervised techniques like consistency regularization and pseudo-labeling to improve model accuracy with partially labeled data across clients.

Contribution

It unifies consistency regularization and pseudo-labeling in federated learning, addressing the challenge of limited labeled data in cross-silo settings.

Findings

SemiFed outperforms baseline methods on image benchmarks.

Effective in both homogeneous and heterogeneous data distributions.

Utilizes model agreement for high-confidence pseudo-labels.

Abstract

Federated learning enables multiple clients, such as mobile phones and organizations, to collaboratively learn a shared model for prediction while protecting local data privacy. However, most recent research and applications of federated learning assume that all clients have fully labeled data, which is impractical in real-world settings. In this work, we focus on a new scenario for cross-silo federated learning, where data samples of each client are partially labeled. We borrow ideas from semi-supervised learning methods where a large amount of unlabeled data is utilized to improve the model's accuracy despite limited access to labeled examples. We propose a new framework dubbed SemiFed that unifies two dominant approaches for semi-supervised learning: consistency regularization and pseudo-labeling. SemiFed first applies advanced data augmentation techniques to enforce consistency regularization and then generates pseudo-labels using the model's predictions during training. SemiFed takes advantage of the federation so that for a given image, the pseudo-label holds only if multiple models from different clients produce a high-confidence prediction and agree on the same label. Extensive experiments on two image benchmarks demonstrate the effectiveness of our approach under both homogeneous and heterogeneous data distribution settings