Differential Privacy Analysis of Decentralized Gossip Averaging under Varying Threat Models

TL;DR

This paper develops a framework for analyzing differential privacy guarantees in decentralized gossip algorithms, showing that privacy leakage grows linearly with training rounds and that private decentralized learning achieves similar excess risk as centralized methods.

Contribution

It introduces an analytical linear systems framework for DP analysis in decentralized gossip algorithms, demonstrating Gaussian mechanism guarantees and asymptotic similarity to centralized private learning.

Findings

DP guarantees are Gaussian mechanism-based.

Sensitivity grows asymptotically as O(T).

Decentralized private learning has similar excess risk to centralized methods.

Abstract

Achieving differential privacy (DP) guarantees in fully decentralized machine learning is challenging due to the absence of a central aggregator and varying trust assumptions among nodes. We present a framework for DP analysis of decentralized gossip-based averaging algorithms with additive node-level noise, from arbitrary views of nodes in a graph. We present an analytical framework based on a linear systems formulation that accurately characterizes privacy leakage between nodes. Our main contribution is showing that the DP guarantees are those of a Gaussian mechanism, where the growth of the squared sensitivity is asymptotically $O(T)$, where $T$ is the number of training rounds, similarly as in the case of central aggregation. As an application of the sensitivity analysis, we show that the excess risk of decentralized private learning for strongly convex losses is asymptotically similar as in centralized private learning.