Better Differentially Private Approximate Histograms and Heavy Hitters using the Misra-Gries Sketch

TL;DR

This paper introduces an improved differentially private mechanism for approximate histograms and heavy hitters using the Misra-Gries sketch, reducing noise dependence on sketch size and enhancing practicality.

Contribution

It presents a novel privacy mechanism that adds noise independent of sketch size, matching non-private accuracy, and introduces a simple post-processing step to further reduce noise.

Findings

Noise magnitude is independent of sketch size.

Maximum error matches the best non-private setting.

Enhanced privacy with less noise for multiple contributions.

Abstract

We consider the problem of computing differentially private approximate histograms and heavy hitters in a stream of elements. In the non-private setting, this is often done using the sketch of Misra and Gries [Science of Computer Programming, 1982]. Chan, Li, Shi, and Xu [PETS 2012] describe a differentially private version of the Misra-Gries sketch, but the amount of noise it adds can be large and scales linearly with the size of the sketch; the more accurate the sketch is, the more noise this approach has to add. We present a better mechanism for releasing a Misra-Gries sketch under $(\varepsilon,\delta)$-differential privacy. It adds noise with magnitude independent of the size of the sketch; in fact, the maximum error coming from the noise is the same as the best known in the private non-streaming setting, up to a constant factor. Our mechanism is simple and likely to be practical. We also give a simple post-processing step of the Misra-Gries sketch that does not increase the worst-case error guarantee. It is sufficient to add noise to this new sketch with less than twice the magnitude of the non-streaming setting. This improves on the previous result for $\varepsilon$-differential privacy where the noise scales linearly to the size of the sketch. Finally, we consider a general setting where users can contribute multiple distinct elements. We present a new sketch with maximum error matching the Misra-Gries sketch. For many parameters in this setting our sketch can be released with less noise under $(\varepsilon, \delta)$-differential privacy.