A survey on secure decentralized optimization and learning

TL;DR

This survey reviews recent advancements in secure decentralized optimization and learning, emphasizing privacy-preserving cryptographic methods and resilient algorithms to mitigate security risks in large-scale distributed systems.

Contribution

It provides a comprehensive overview of cryptographic tools and resilient protocols for enhancing security and privacy in decentralized optimization and learning.

Findings

Cryptographic tools enable privacy-preserving decentralized algorithms.

Resilient protocols improve robustness against malicious agents.

Current trends highlight integration of security methods in large-scale systems.

Abstract

Decentralized optimization has become a standard paradigm for solving large-scale decision-making problems and training large machine learning models without centralizing data. However, this paradigm introduces new privacy and security risks, with malicious agents potentially able to infer private data or impair the model accuracy. Over the past decade, significant advancements have been made in developing secure decentralized optimization and learning frameworks and algorithms. This survey provides a comprehensive tutorial on these advancements. We begin with the fundamentals of decentralized optimization and learning, highlighting centralized aggregation and distributed consensus as key modules exposed to security risks in federated and distributed optimization, respectively. Next, we focus on privacy-preserving algorithms, detailing three cryptographic tools and their integration into decentralized optimization and learning systems. Additionally, we examine resilient algorithms, exploring the design and analysis of resilient aggregation and consensus protocols that support these systems. We conclude the survey by discussing current trends and potential future directions.