Private Online Learning via Lazy Algorithms

TL;DR

This paper introduces a transformation for lazy online learning algorithms to achieve differential privacy, resulting in improved regret bounds for private online prediction tasks, especially in high privacy regimes.

Contribution

The authors propose a novel transformation that converts lazy online learning algorithms into differentially private algorithms, achieving near-optimal regret bounds in private online learning.

Findings

Achieves regret bounds of 0   T^{1/3}        for DP-OPE.

Achieves regret bounds of 0   T^{1/3}        for DP-OCO.

Provides a lower bound showing the optimality of these regret rates.

Abstract

We study the problem of private online learning, specifically, online prediction from experts (OPE) and online convex optimization (OCO). We propose a new transformation that transforms lazy online learning algorithms into private algorithms. We apply our transformation for differentially private OPE and OCO using existing lazy algorithms for these problems. Our final algorithms obtain regret, which significantly improves the regret in the high privacy regime $\varepsilon \ll 1$, obtaining $\sqrt{T \log d} + T^{1/3} \log(d)/\varepsilon^{2/3}$ for DP-OPE and $\sqrt{T} + T^{1/3} \sqrt{d}/\varepsilon^{2/3}$ for DP-OCO. We also complement our results with a lower bound for DP-OPE, showing that these rates are optimal for a natural family of low-switching private algorithms.