FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels

TL;DR

FedDiv introduces a global noise filtering approach for federated learning with noisy labels, leveraging cross-client knowledge to improve noise detection, training stability, and overall model performance without compromising data privacy.

Contribution

The paper proposes FedDiv, a novel federated noise filtering framework that models global label noise distribution and enhances local data credibility assessment, outperforming existing methods.

Findings

FedDiv outperforms state-of-the-art F-LNL methods on multiple datasets.

It effectively identifies noisy labels across clients without data sharing.

The approach improves training stability and model accuracy under various noise conditions.

Abstract

Federated learning with noisy labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning by aggregating multiple client models trained with local noisy or clean samples. On the basis of a federated learning framework, recent advances primarily adopt label noise filtering to separate clean samples from noisy ones on each client, thereby mitigating the negative impact of label noise. However, these prior methods do not learn noise filters by exploiting knowledge across all clients, leading to sub-optimal and inferior noise filtering performance and thus damaging training stability. In this paper, we present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter for effectively identifying samples with noisy labels on every client, thereby raising stability during local training sessions. Without sacrificing data privacy, this is achieved by modeling the global distribution of label noise across all clients. Then, in an effort to make the global model achieve higher performance, we introduce a Predictive Consistency based Sampler to identify more credible local data for local model training, thus preventing noise memorization and further boosting the training stability. Extensive experiments on CIFAR-10, CIFAR-100, and Clothing1M demonstrate that \texttt{FedDiv} achieves superior performance over state-of-the-art F-LNL methods under different label noise settings for both IID and non-IID data partitions. Source code is publicly available at https://github.com/lijichang/FLNL-FedDiv.