Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels

TL;DR

This paper introduces FedGR, a method to improve federated learning robustness against noisy labels by leveraging the slow memorization phenomenon of global models, outperforming existing methods.

Contribution

The paper proposes FedGR, a novel approach that rectifies noisy labels and regularizes local training in federated learning, addressing label noise and heterogeneity.

Findings

FedGR outperforms seven state-of-the-art baselines in noisy label scenarios.

Global models in FL exhibit slow memorization of noisy labels, aiding robustness.

Extensive experiments on three benchmarks validate FedGR's effectiveness.

Abstract

Conventioanl federated learning (FL) heavily depends on high-quality labels, which are often impractical in the real world, leading to the federated label-noise (F-LN) problem. Worsely, the F-LN problem is exacerbated by the heterogeneity of FL, whereas clients experience different labelnoise types, ratios, and data distribution. In this study, we first observe an intriguing phenomenon that the global model of FL exhibits a slow memorization of noisy labels, suggesting its ability to maintain reliable predictions and robust representations in FL. Motivated on this, we propose a novel method termed Federated Global Reviser (FedGR), a straightforward yet effective method comprising three modules that collaboratively rectify noisy labels and regularize local training. By exploiting above inherent property, FedGR improve the label-noise robustness of FL in a self-contained manner. Extensive experiments on three widely used F-LN benchmarks demonstrate the superior performance of FedGR, consistently outperforming seven state-of-the-art baselines even in severe label-noise and data heterogeneity. Code will be released as soon as possible.