Private Prediction via Shrinkage

TL;DR

This paper improves the efficiency of differentially private prediction algorithms in streaming settings by reducing the labeled sample complexity from square root to polylogarithmic dependence on the number of queries, for both oblivious and adaptive adversaries.

Contribution

It introduces new private prediction algorithms with polylogarithmic query dependence, significantly reducing labeled sample complexity in streaming scenarios.

Findings

Achieved polylogarithmic dependence on number of queries for oblivious adversaries.

Provided sample complexity bounds for halfspaces over ^d.

Enhanced privacy guarantees in streaming prediction models.

Abstract

We study differentially private prediction introduced by Dwork and Feldman (COLT 2018): an algorithm receives one labeled sample set $S$ and then answers a stream of unlabeled queries while the output transcript remains $(\varepsilon,\delta)$-differentially private with respect to $S$. Standard composition yields a $\sqrt{T}$ dependence for $T$ queries. We show that this dependence can be reduced to polylogarithmic in $T$ in streaming settings. For an oblivious online adversary and any concept class $\mathcal{C}$, we give a private predictor that answers $T$ queries with $|S|= \tilde{O}(VC(\mathcal{C})^{3.5}\log^{3.5}T)$ labeled examples. For an adaptive online adversary and halfspaces over $\mathbb{R}^d$, we obtain $|S|=\tilde{O}\left(d^{5.5}\log T\right)$.