Privacy Attacks in Decentralized Learning

TL;DR

This paper reveals that decentralized gradient descent protocols are vulnerable to reconstruction attacks, enabling users to infer private data of others beyond their immediate neighbors, challenging assumptions of privacy in such systems.

Contribution

First attack demonstrated against D-GD that reconstructs private data of non-neighbor users, highlighting privacy vulnerabilities in decentralized learning.

Findings

Attack effective on real graphs and datasets.

Number of compromised users can be large with few attackers.

Graph topology and attacker position influence attack success.

Abstract

Decentralized Gradient Descent (D-GD) allows a set of users to perform collaborative learning without sharing their data by iteratively averaging local model updates with their neighbors in a network graph. The absence of direct communication between non-neighbor nodes might lead to the belief that users cannot infer precise information about the data of others. In this work, we demonstrate the opposite, by proposing the first attack against D-GD that enables a user (or set of users) to reconstruct the private data of other users outside their immediate neighborhood. Our approach is based on a reconstruction attack against the gossip averaging protocol, which we then extend to handle the additional challenges raised by D-GD. We validate the effectiveness of our attack on real graphs and datasets, showing that the number of users compromised by a single or a handful of attackers is often surprisingly large. We empirically investigate some of the factors that affect the performance of the attack, namely the graph topology, the number of attackers, and their position in the graph.