Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learning

TL;DR

This paper demonstrates that the weighted mean aggregator in decentralized learning can be more robust to label poisoning attacks than robust aggregators under certain conditions, especially considering network topology and data heterogeneity.

Contribution

It provides a theoretical analysis showing the topology-independent robustness of the weighted mean and identifies scenarios where it outperforms robust aggregators.

Findings

Weighted mean can outperform robust aggregators under high heterogeneity.

Robustness of aggregators depends on network topology and contamination rates.

Empirical results confirm theoretical predictions about topology's role in robustness.

Abstract

Robustness to malicious attacks is crucial for practical decentralized signal processing and machine learning systems. A typical example of such attacks is label poisoning, meaning that some agents possess corrupted local labels and share models trained on these poisoned data. To defend against malicious attacks, existing works often focus on designing robust aggregators; meanwhile, the weighted mean aggregator is typically considered a simple, vulnerable baseline. This paper analyzes the robustness of decentralized gradient descent under label poisoning attacks, considering both robust and weighted mean aggregators. Theoretical results reveal that the learning errors of robust aggregators depend on the network topology, whereas the performance of weighted mean aggregator is topology-independent. Remarkably, the weighted mean aggregator, although often considered vulnerable, can outperform robust aggregators under sufficient heterogeneity, particularly when: (i) the global contamination rate (i.e., the fraction of poisoned agents for the entire network) is smaller than the local contamination rate (i.e., the maximal fraction of poisoned neighbors for the regular agents); (ii) the network of regular agents is disconnected; or (iii) the network of regular agents is sparse and the local contamination rate is high. Empirical results support our theoretical findings, highlighting the important role of network topology in the robustness to label poisoning attacks.