# On Differentially Private Online Predictions

## TL;DR

This paper introduces an interactive variant of joint differential privacy tailored for online processes, demonstrating its favorable properties and showing that private online learning can be achieved with only polynomial overhead in mistake bounds.

## Contribution

It proposes a new interactive joint differential privacy definition and proves that it allows private online learning with polynomial mistake overhead, unlike previous more restrictive notions.

## Key findings

- Interactive joint privacy satisfies group privacy, composition, and post-processing.
- Any online learning rule can be privatized with polynomial mistake overhead.
- Contrasts with previous notions requiring double exponential overhead.

## Abstract

In this work we introduce an interactive variant of joint differential privacy towards handling online processes in which existing privacy definitions seem too restrictive. We study basic properties of this definition and demonstrate that it satisfies (suitable variants) of group privacy, composition, and post processing. We then study the cost of interactive joint privacy in the basic setting of online classification. We show that any (possibly non-private) learning rule can be effectively transformed to a private learning rule with only a polynomial overhead in the mistake bound. This demonstrates a stark difference with more restrictive notions of privacy such as the one studied by Golowich and Livni (2021), where only a double exponential overhead on the mistake bound is known (via an information theoretic upper bound).