Collaboratively Learning Federated Models from Noisy Decentralized Data

TL;DR

This paper introduces FedNS, a noise-aware federated learning method that detects and mitigates the impact of noisy data from clients, significantly improving global model performance in decentralized settings.

Contribution

It proposes a novel client input assessment in gradient space and a plug-in aggregation method, FedNS, to effectively handle noisy data in federated learning.

Findings

FedNS improves model accuracy by up to 15.85% in non-IID settings.

Gradient norm distribution disparity helps identify low-quality client data.

The approach enhances existing FL strategies with minimal additional complexity.

Abstract

Federated learning (FL) has emerged as a prominent method for collaboratively training machine learning models using local data from edge devices, all while keeping data decentralized. However, accounting for the quality of data contributed by local clients remains a critical challenge in FL, as local data are often susceptible to corruption by various forms of noise and perturbations, which compromise the aggregation process and lead to a subpar global model. In this work, we focus on addressing the problem of noisy data in the input space, an under-explored area compared to the label noise. We propose a comprehensive assessment of client input in the gradient space, inspired by the distinct disparity observed between the density of gradient norm distributions of models trained on noisy and clean input data. Based on this observation, we introduce a straightforward yet effective approach to identify clients with low-quality data at the initial stage of FL. Furthermore, we propose a noise-aware FL aggregation method, namely Federated Noise-Sifting (FedNS), which can be used as a plug-in approach in conjunction with widely used FL strategies. Our extensive evaluation on diverse benchmark datasets under different federated settings demonstrates the efficacy of FedNS. Our method effortlessly integrates with existing FL strategies, enhancing the global model's performance by up to 13.68% in IID and 15.85% in non-IID settings when learning from noisy decentralized data.