Differentially Private Online-to-Batch for Smooth Losses

TL;DR

This paper introduces a new reduction method that transforms online convex optimization algorithms into differentially private stochastic convex optimization algorithms with optimal convergence rates for smooth losses, maintaining linear time complexity.

Contribution

It presents a novel online-to-batch reduction technique that achieves optimal convergence rates for differentially private convex optimization on smooth losses, extending to adaptive algorithms.

Findings

Achieves optimal convergence rate of  O(1/  T +     d/   T) for smooth losses.

Provides a linear-time algorithm for differentially private stochastic convex optimization.

Extends to adaptive algorithms with data-dependent convergence rates.

Abstract

We develop a new reduction that converts any online convex optimization algorithm suffering $O(\sqrt{T})$ regret into an $\epsilon$-differentially private stochastic convex optimization algorithm with the optimal convergence rate $\tilde O(1/\sqrt{T} + \sqrt{d}/\epsilon T)$ on smooth losses in linear time, forming a direct analogy to the classical non-private "online-to-batch" conversion. By applying our techniques to more advanced adaptive online algorithms, we produce adaptive differentially private counterparts whose convergence rates depend on apriori unknown variances or parameter norms.