Differentially Private and Communication-Efficient Distributed Nonconvex Optimization Algorithms

TL;DR

This paper introduces differentially private distributed algorithms for nonconvex optimization that are communication-efficient, achieving sublinear or linear convergence depending on the cost function's properties, and ensuring privacy without relying on specific compressor forms.

Contribution

The paper proposes novel privacy-preserving distributed algorithms that work with compressed communication and achieve convergence guarantees for nonconvex functions, without relying on specific compressor assumptions.

Findings

Achieve sublinear convergence for smooth nonconvex functions

Achieve linear convergence under Polyak-Łojasiewicz condition

Guarantee $$-differential privacy without compressor-specific analysis

Abstract

This paper studies the privacy-preserving distributed optimization problem under limited communication, where each agent aims to keep its cost function private while minimizing the sum of all agents' cost functions. To this end, we propose two differentially private distributed algorithms under compressed communication. We show that the proposed algorithms achieve sublinear convergence for smooth (possibly nonconvex) cost functions and linear convergence when the global cost function additionally satisfies the Polyak-{\L}ojasiewicz condition, even for a general class of compressors with bounded relative compression error. Furthermore, we rigorously prove that the proposed algorithms ensure $\epsilon$-differential privacy. Unlike methods in the literature, the analysis of privacy under the proposed algorithms do not rely on the specific forms of compressors. Simulations are presented to demonstrate the effectiveness of our proposed approach.