The Built-In Robustness of Decentralized Federated Averaging to Bad Data

TL;DR

Decentralized federated averaging demonstrates surprising robustness to localized bad data, with the averaging process preventing any single node from disproportionately affecting the overall model, even under severe data corruption scenarios.

Contribution

This paper reveals that decentralized federated averaging inherently resists the influence of corrupted data, a counterintuitive finding that enhances understanding of its robustness without central oversight.

Findings

Decentralized FedAvg is robust to localized bad data.

Robustness increases when corrupted data is concentrated on a single node.

Averaging prevents any node from dominating the learning process.

Abstract

Decentralized federated learning (DFL) enables devices to collaboratively train models over complex network topologies without relying on a central controller. In this setting, local data remains private, but its quality and quantity can vary significantly across nodes. The extent to which a fully decentralized system is vulnerable to poor-quality or corrupted data remains unclear, but several factors could contribute to potential risks. Without a central authority, there can be no unified mechanism to detect or correct errors, and each node operates with a localized view of the data distribution, making it difficult for the node to assess whether its perspective aligns with the true distribution. Moreover, models trained on low-quality data can propagate through the network, amplifying errors. To explore the impact of low-quality data on DFL, we simulate two scenarios with degraded data quality -- one where the corrupted data is evenly distributed in a subset of nodes and one where it is concentrated on a single node -- using a decentralized implementation of FedAvg. Our results reveal that averaging-based decentralized learning is remarkably robust to localized bad data, even when the corrupted data resides in the most influential nodes of the network. Counterintuitively, this robustness is further enhanced when the corrupted data is concentrated on a single node, regardless of its centrality in the communication network topology. This phenomenon is explained by the averaging process, which ensures that no single node -- however central -- can disproportionately influence the overall learning process.