Differentially Private Decentralized Optimization with Relay Communication

TL;DR

This paper introduces DP-RECAL, a differentially private decentralized optimization algorithm that reduces privacy leakage and communication costs using relay communication, with proven convergence and robustness against attacks.

Contribution

The paper proposes DP-RECAL, a novel algorithm combining relay communication and operator splitting to enhance privacy and efficiency in decentralized optimization.

Findings

DP-RECAL achieves superior privacy performance compared to existing algorithms.

It reduces communication complexity while maintaining convergence guarantees.

The algorithm effectively defends against classical gradient leakage attacks.

Abstract

Security concerns in large-scale networked environments are becoming increasingly critical. To further improve the algorithm security from the design perspective of decentralized optimization algorithms, we introduce a new measure: Privacy Leakage Frequency (PLF), which reveals the relationship between communication and privacy leakage of algorithms, showing that lower PLF corresponds to lower privacy budgets. Based on such assertion, a novel differentially private decentralized primal--dual algorithm named DP-RECAL is proposed to take advantage of operator splitting method and relay communication mechanism to experience less PLF so as to reduce the overall privacy budget. To the best of our knowledge, compared with existing differentially private algorithms, DP-RECAL presents superior privacy performance and communication complexity. In addition, with uncoordinated network-independent stepsizes, we prove the convergence of DP-RECAL for general convex problems and establish a linear convergence rate under the metric subregularity. Evaluation analysis on least squares problem and numerical experiments on real-world datasets verify our theoretical results and demonstrate that DP-RECAL can defend some classical gradient leakage attacks.