Enhancing Privacy in Decentralized Min-Max Optimization: A Differentially Private Approach

TL;DR

This paper introduces DPMixSGD, a novel differentially private algorithm for decentralized non-convex min-max optimization, effectively balancing privacy preservation with convergence performance in multi-agent systems.

Contribution

It presents DPMixSGD, the first privacy-preserving algorithm tailored for decentralized non-convex min-max problems, building on STORM and providing theoretical convergence guarantees.

Findings

DPMixSGD maintains convergence despite added noise.

Theoretical bounds confirm privacy without significant performance loss.

Experimental results validate effectiveness across tasks.

Abstract

Decentralized min-max optimization allows multi-agent systems to collaboratively solve global min-max optimization problems by facilitating the exchange of model updates among neighboring agents, eliminating the need for a central server. However, sharing model updates in such systems carry a risk of exposing sensitive data to inference attacks, raising significant privacy concerns. To mitigate these privacy risks, differential privacy (DP) has become a widely adopted technique for safeguarding individual data. Despite its advantages, implementing DP in decentralized min-max optimization poses challenges, as the added noise can hinder convergence, particularly in non-convex scenarios with complex agent interactions in min-max optimization problems. In this work, we propose an algorithm called DPMixSGD (Differential Private Minmax Hybrid Stochastic Gradient Descent), a novel privacy-preserving algorithm specifically designed for non-convex decentralized min-max optimization. Our method builds on the state-of-the-art STORM-based algorithm, one of the fastest decentralized min-max solutions. We rigorously prove that the noise added to local gradients does not significantly compromise convergence performance, and we provide theoretical bounds to ensure privacy guarantees. To validate our theoretical findings, we conduct extensive experiments across various tasks and models, demonstrating the effectiveness of our approach.