Characterizing the Accuracy-Communication-Privacy Trade-off in Distributed Stochastic Convex Optimization

TL;DR

This paper thoroughly analyzes the interplay between accuracy, communication efficiency, and privacy in distributed differentially private stochastic convex optimization, providing theoretical bounds and a new algorithm to characterize the fundamental trade-offs.

Contribution

It introduces a novel lower bound and a new algorithm based on Vaidya's method, fully characterizing the accuracy-communication-privacy trade-off in distributed DP-SCO.

Findings

Established matching lower and upper bounds for the trade-off.

Proposed a new distributed DP-SCO algorithm based on Vaidya's plane cutting method.

Provided a complete theoretical characterization of the trade-offs.

Abstract

We consider the problem of differentially private stochastic convex optimization (DP-SCO) in a distributed setting with $M$ clients, where each of them has a local dataset of $N$ i.i.d. data samples from an underlying data distribution. The objective is to design an algorithm to minimize a convex population loss using a collaborative effort across $M$ clients, while ensuring the privacy of the local datasets. In this work, we investigate the accuracy-communication-privacy trade-off for this problem. We establish matching converse and achievability results using a novel lower bound and a new algorithm for distributed DP-SCO based on Vaidya's plane cutting method. Thus, our results provide a complete characterization of the accuracy-communication-privacy trade-off for DP-SCO in the distributed setting.