Source Anonymity for Private Random Walk Decentralized Learning

TL;DR

This paper introduces a privacy-preserving decentralized learning algorithm using cryptography and anonymization, ensuring source anonymity in random walk-based model updates on networks.

Contribution

It proposes a novel source anonymity scheme for decentralized learning that provides theoretical guarantees, especially on random regular graphs.

Findings

Achieves source anonymity with provable guarantees.

Designs a network-dependent probability distribution for source hiding.

Focuses on random regular graphs for rigorous analysis.

Abstract

This paper considers random walk-based decentralized learning, where at each iteration of the learning process, one user updates the model and sends it to a randomly chosen neighbor until a convergence criterion is met. Preserving data privacy is a central concern and open problem in decentralized learning. We propose a privacy-preserving algorithm based on public-key cryptography and anonymization. In this algorithm, the user updates the model and encrypts the result using a distant user's public key. The encrypted result is then transmitted through the network with the goal of reaching that specific user. The key idea is to hide the source's identity so that, when the destination user decrypts the result, it does not know who the source was. The challenge is to design a network-dependent probability distribution (at the source) over the potential destinations such that, from the receiver's perspective, all users have a similar likelihood of being the source. We introduce the problem and construct a scheme that provides anonymity with theoretical guarantees. We focus on random regular graphs to establish rigorous guarantees.