Federated Learning with Instance-Dependent Noisy Label

TL;DR

This paper introduces FedBeat, a novel federated learning algorithm designed to effectively handle instance-dependent noisy labels by estimating transition matrices and correcting classifiers, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes FedBeat, a new federated learning approach that estimates transition matrices for instance-dependent noise, improving robustness and accuracy in noisy label scenarios.

Findings

FedBeat significantly outperforms state-of-the-art methods on CIFAR-10 and SVHN.

The method effectively estimates transition matrices in a federated setting.

FedBeat enhances classifier performance under complex noisy label conditions.

Abstract

Federated learning (FL) with noisy labels poses a significant challenge. Existing methods designed for handling noisy labels in centralized learning tend to lose their effectiveness in the FL setting, mainly due to the small dataset size and the heterogeneity of client data. While some attempts have been made to tackle FL with noisy labels, they primarily focused on scenarios involving class-conditional noise. In this paper, we study the more challenging and practical issue of instance-dependent noise (IDN) in FL. We introduce a novel algorithm called FedBeat (Federated Learning with Bayesian Ensemble-Assisted Transition Matrix Estimation). FedBeat aims to build a global statistically consistent classifier using the IDN transition matrix (IDNTM), which encompasses three synergistic steps: (1) A federated data extraction step that constructs a weak global model and extracts high-confidence data using a Bayesian model ensemble method. (2) A federated transition matrix estimation step in which clients collaboratively train an IDNTM estimation network based on the extracted data. (3) A federated classifier correction step that enhances the global model's performance by training it using a loss function tailored for noisy labels, leveraging the IDNTM. Experiments conducted on CIFAR-10 and SVHN verify that the proposed method significantly outperforms state-of-the-art methods.