Differentially Private Perturbed Push-Sum Protocol and Its Application in Non-Convex Optimization

TL;DR

This paper introduces DPPS, a lightweight protocol for differential privacy in decentralized networks, and applies it to non-convex optimization with improved privacy-utility trade-offs and convergence guarantees.

Contribution

The paper proposes DPPS, a general protocol-level differential privacy method with a novel sensitivity estimation, and develops PartPSP, a decentralized non-convex optimization algorithm using partial communication for better performance.

Findings

DPPS provides rigorous differential privacy with minimal communication overhead.

PartPSP achieves improved optimization performance under the same privacy constraints.

Experimental results confirm the effectiveness and superiority of the proposed methods.

Abstract

In decentralized networks, nodes cannot ensure that their shared information will be securely preserved by their neighbors, making privacy vulnerable to inference by curious nodes. Adding calibrated random noise before communication to satisfy differential privacy offers a proven defense; however, most existing methods are tailored to specific downstream tasks and lack a general, protocol-level privacy-preserving solution. To bridge this gap, we propose Differentially Private Perturbed Push-Sum (DPPS), a lightweight differential privacy protocol for decentralized communication. Since protocol-level differential privacy introduces the unique challenge of obtaining the sensitivity for each communication round, DPPS introduces a novel sensitivity estimation mechanism that requires each node to compute and broadcast only one scalar per round, enabling rigorous differential privacy guarantees. This design allows DPPS to serve as a plug-and-play, low-cost privacy-preserving solution for downstream applications built on it. To provide a concrete instantiation of DPPS and better balance the privacy-utility trade-off, we design PartPSP, a privacy-preserving decentralized algorithm for non-convex optimization that integrates a partial communication mechanism. By partitioning model parameters into local and shared components and applying DPPS only to the shared parameters, PartPSP reduces the dimensionality of consensus data, thereby lowering the magnitude of injected noise and improving optimization performance. We theoretically prove that PartPSP converges under non-convex objectives and, with partial communication, achieves better optimization performance under the same privacy budget. Experimental results validate the effectiveness of DPPS's privacy-preserving and demonstrate that PartPSP outperforms existing privacy-preserving decentralized optimization algorithms.