Can Decentralized Learning be more robust than Federated Learning?

TL;DR

This paper investigates the robustness of decentralized learning compared to federated learning, introducing new attacks that compromise decentralized protocols and demonstrating their increased vulnerability.

Contribution

The paper presents two novel attacks on decentralized learning protocols and proves that decentralization inherently offers less robustness than federated learning.

Findings

New attacks enable Byzantine users to manipulate models and exclude participants.

Decentralized learning is always less robust than federated learning against Byzantine attacks.

Abstract

Decentralized Learning (DL) is a peer--to--peer learning approach that allows a group of users to jointly train a machine learning model. To ensure correctness, DL should be robust, i.e., Byzantine users must not be able to tamper with the result of the collaboration. In this paper, we introduce two \textit{new} attacks against DL where a Byzantine user can: make the network converge to an arbitrary model of their choice, and exclude an arbitrary user from the learning process. We demonstrate our attacks' efficiency against Self--Centered Clipping, the state--of--the--art robust DL protocol. Finally, we show that the capabilities decentralization grants to Byzantine users result in decentralized learning \emph{always} providing less robustness than federated learning.