Local Graph-homomorphic Processing for Privatized Distributed Systems

TL;DR

This paper introduces local graph-homomorphic processing, a method for generating dependent noise in distributed systems that ensures differential privacy without degrading model performance, improving upon previous unstructured noise addition techniques.

Contribution

The paper presents a novel structured noise generation method over network edges that preserves privacy and maintains model accuracy in distributed learning.

Findings

Noise does not affect the learned model's performance.

Structured noise respects graph topology, unlike previous methods.

Method applied to linear regression over networks.

Abstract

We study the generation of dependent random numbers in a distributed fashion in order to enable privatized distributed learning by networked agents. We propose a method that we refer to as local graph-homomorphic processing; it relies on the construction of particular noises over the edges to ensure a certain level of differential privacy. We show that the added noise does not affect the performance of the learned model. This is a significant improvement to previous works on differential privacy for distributed algorithms, where the noise was added in a less structured manner without respecting the graph topology and has often led to performance deterioration. We illustrate the theoretical results by considering a linear regression problem over a network of agents.