Privacy-Preserving Push-Pull Method for Decentralized Optimization via State Decomposition

TL;DR

This paper introduces PPSD, a privacy-preserving decentralized optimization algorithm that uses state decomposition to protect private information without sacrificing accuracy, achieving linear convergence for smooth, strongly convex functions.

Contribution

The paper proposes a novel PPSD algorithm combining gradient tracking with state decomposition, enhancing privacy without heavy computational costs in decentralized optimization.

Findings

Achieves R-linear convergence rate for strongly convex, smooth functions.

Ensures privacy against honest-but-curious neighbors.

Validated through simulations confirming theoretical results.

Abstract

Distributed optimization is manifesting great potential in multiple fields, e.g., machine learning, control, and resource allocation. Existing decentralized optimization algorithms require sharing explicit state information among the agents, which raises the risk of private information leakage. To ensure privacy security, combining information security mechanisms, such as differential privacy and homomorphic encryption, with traditional decentralized optimization algorithms is a commonly used means. However, this would either sacrifice optimization accuracy or incur heavy computational burden. To overcome these shortcomings, we develop a novel privacy-preserving decentralized optimization algorithm, called PPSD, that combines gradient tracking with a state decomposition mechanism. Specifically, each agent decomposes its state associated with the gradient into two substates. One substate is used for interaction with neighboring agents, and the other substate containing private information acts only on the first substate and thus is entirely agnostic to other agents. For the strongly convex and smooth objective functions, PPSD attains a $R$-linear convergence rate. Moreover, the algorithm can preserve the agents' private information from being leaked to honest-but-curious neighbors. Simulations further confirm the results.