FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysis

TL;DR

FedIL introduces a federated semi-supervised learning framework that leverages unlabeled client data and a small labeled server dataset, employing iterative similarity fusion and incremental confidence to improve model convergence and performance.

Contribution

This paper proposes FedIL, a novel federated incremental learning framework that effectively utilizes unlabeled client data and a small labeled server dataset with convergence guarantees.

Findings

FedIL accelerates model convergence using cosine similarity.

The framework effectively leverages unlabeled data for semi-supervised federated learning.

Theoretical proof of convergence based on Banach Fixed Point Theorem.

Abstract

Most existing federated learning methods assume that clients have fully labeled data to train on, while in reality, it is hard for the clients to get task-specific labels due to users' privacy concerns, high labeling costs, or lack of expertise. This work considers the server with a small labeled dataset and intends to use unlabeled data in multiple clients for semi-supervised learning. We propose a new framework with a generalized model, Federated Incremental Learning (FedIL), to address the problem of how to utilize labeled data in the server and unlabeled data in clients separately in the scenario of Federated Learning (FL). FedIL uses the Iterative Similarity Fusion to enforce the server-client consistency on the predictions of unlabeled data and uses incremental confidence to establish a credible pseudo-label set in each client. We show that FedIL will accelerate model convergence by Cosine Similarity with normalization, proved by Banach Fixed Point Theorem. The code is available at https://anonymous.4open.science/r/fedil.