Privacy-Preserving Distributed Optimization and Learning

TL;DR

This survey reviews privacy-preserving techniques in distributed optimization and learning, emphasizing differential privacy as a promising approach due to its efficiency and effectiveness in protecting sensitive information.

Contribution

It provides a comprehensive overview of privacy-preserving methods, compares their advantages and disadvantages, and introduces differential privacy algorithms suitable for high-dimensional applications.

Findings

Differential privacy offers a promising balance between privacy and accuracy.

Several algorithms demonstrate effective privacy preservation in machine learning tasks.

Challenges and future directions are identified for advancing privacy-preserving distributed optimization.

Abstract

Distributed optimization and learning has recently garnered great attention due to its wide applications in sensor networks, smart grids, machine learning, and so forth. Despite rapid development, existing distributed optimization and learning algorithms require each agent to exchange messages with its neighbors, which may expose sensitive information and raise significant privacy concerns. In this survey paper, we overview privacy-preserving distributed optimization and learning methods. We first discuss cryptography, differential privacy, and other techniques that can be used for privacy preservation and indicate their pros and cons for privacy protection in distributed optimization and learning. We believe that among these approaches, differential privacy is most promising due to its low computational and communication complexities, which are extremely appealing for modern learning based applications with high dimensions of optimization variables. We then introduce several differential-privacy algorithms that can simultaneously ensure privacy and optimization accuracy. Moreover, we provide example applications in several machine learning problems to confirm the real-world effectiveness of these algorithms. Finally, we highlight some challenges in this research domain and discuss future directions.