Privacy-Preserving Distributed Online Mirror Descent for Nonconvex Optimization

TL;DR

This paper introduces a novel distributed online mirror descent algorithm that ensures differential privacy for nonconvex optimization over dynamic networks, achieving sublinear regret growth.

Contribution

It presents the first privacy-preserving distributed online mirror descent method applicable to nonconvex functions with theoretical privacy and regret guarantees.

Findings

Guarantees $ ext{ε}$-differential privacy at each iteration.

Achieves sublinear regret growth of $O(\sqrt{T})$ over time.

Demonstrates effectiveness through numerical simulations.

Abstract

We investigate the distributed online nonconvex optimization problem with differential privacy over time-varying networks. Each node minimizes the sum of several nonconvex functions while preserving the node's differential privacy. We propose a privacy-preserving distributed online mirror descent algorithm for nonconvex optimization, which uses the mirror descent to update decision variables and the Laplace differential privacy mechanism to protect privacy. Unlike the existing works, the proposed algorithm allows the cost functions to be nonconvex, which is more applicable. Based upon these, we prove that if the communication network is $B$-strongly connected and the constraint set is compact, then by choosing the step size properly, the algorithm guarantees $\epsilon$-differential privacy at each time. Furthermore, we prove that if the local cost functions are $\beta$-smooth, then the regret over time horizon $T$ grows sublinearly while preserving differential privacy, with an upper bound $O(\sqrt{T})$. Finally, the effectiveness of the algorithm is demonstrated through numerical simulations.