Decentralized Optimization with Amplified Privacy via Efficient Communication

TL;DR

This paper introduces a decentralized optimization algorithm that enhances privacy and communication efficiency by combining random agent activation, sparsified communication, and differential privacy, with proven convergence and empirical validation.

Contribution

It proposes a novel decentralized stochastic gradient descent method that amplifies privacy guarantees while maintaining accuracy through efficient communication strategies.

Findings

Reduced noise in privacy-preserving updates

Improved privacy-accuracy trade-off

Effective communication reduction demonstrated

Abstract

Decentralized optimization is crucial for multi-agent systems, with significant concerns about communication efficiency and privacy. This paper explores the role of efficient communication in decentralized stochastic gradient descent algorithms for enhancing privacy preservation. We develop a novel algorithm that incorporates two key features: random agent activation and sparsified communication. Utilizing differential privacy, we demonstrate that these features reduce noise without sacrificing privacy, thereby amplifying the privacy guarantee and improving accuracy. Additionally, we analyze the convergence and the privacy-accuracy-communication trade-off of the proposed algorithm. Finally, we present experimental results to illustrate the effectiveness of our algorithm.