Towards Federated Learning against Noisy Labels via Local Self-Regularization

TL;DR

This paper introduces a Local Self-Regularization method for federated learning that mitigates the impact of noisy labels while preserving data privacy, demonstrating improved robustness and performance on benchmark and real-world datasets.

Contribution

The paper proposes a novel Local Self-Regularization approach that reduces noisy label effects in federated learning without compromising privacy, outperforming existing methods.

Findings

Achieves notable resistance to noisy labels across various noise levels.

Outperforms state-of-the-art methods on benchmark datasets.

Demonstrates superior performance on the real-world Clothing1M dataset.

Abstract

Federated learning (FL) aims to learn joint knowledge from a large scale of decentralized devices with labeled data in a privacy-preserving manner. However, since high-quality labeled data require expensive human intelligence and efforts, data with incorrect labels (called noisy labels) are ubiquitous in reality, which inevitably cause performance degradation. Although a lot of methods are proposed to directly deal with noisy labels, these methods either require excessive computation overhead or violate the privacy protection principle of FL. To this end, we focus on this issue in FL with the purpose of alleviating performance degradation yielded by noisy labels meanwhile guaranteeing data privacy. Specifically, we propose a Local Self-Regularization method, which effectively regularizes the local training process via implicitly hindering the model from memorizing noisy labels and explicitly narrowing the model output discrepancy between original and augmented instances using self distillation. Experimental results demonstrate that our proposed method can achieve notable resistance against noisy labels in various noise levels on three benchmark datasets. In addition, we integrate our method with existing state-of-the-arts and achieve superior performance on the real-world dataset Clothing1M. The code is available at https://github.com/Sprinter1999/FedLSR.