Locally Private Histograms in All Privacy Regimes

TL;DR

This paper explores the accuracy of locally private histograms across all privacy regimes, especially medium and low privacy levels, providing near-tight bounds and improved protocols with better communication efficiency.

Contribution

It establishes near-tight error bounds for locally private histograms in medium-to-low privacy regimes and introduces a shuffle model protocol with improved message complexity.

Findings

Achieves near-tight bounds on $\,\ell_ extinfty$ error in medium-to-low privacy regimes.

Provides a shuffle model protocol with accuracy comparable to existing methods but with reduced communication.

Empirically compares algorithms across all privacy regimes to evaluate practical performance.

Abstract

Frequency estimation, a.k.a. histograms, is a workhorse of data analysis, and as such has been thoroughly studied under differentially privacy. In particular, computing histograms in the \emph{local} model of privacy has been the focus of a fruitful recent line of work, and various algorithms have been proposed, achieving the order-optimal $\ell_\infty$ error in the high-privacy (small $\varepsilon$) regime while balancing other considerations such as time- and communication-efficiency. However, to the best of our knowledge, the picture is much less clear when it comes to the medium- or low-privacy regime (large $\varepsilon$), despite its increased relevance in practice. In this paper, we investigate locally private histograms, and the very related distribution learning task, in this medium-to-low privacy regime, and establish near-tight (and somewhat unexpected) bounds on the $\ell_\infty$ error achievable. As a direct corollary of our results, we obtain a protocol for histograms in the \emph{shuffle} model of differential privacy, with accuracy matching previous algorithms but significantly better message and communication complexity. Our theoretical findings emerge from a novel analysis, which appears to improve bounds across the board for the locally private histogram problem. We back our theoretical findings by an empirical comparison of existing algorithms in all privacy regimes, to assess their typical performance and behaviour beyond the worst-case setting.