Hadamard Response: Estimating Distributions Privately, Efficiently, and with Little Communication

TL;DR

This paper introduces Hadamard Response, a new local differential privacy scheme for estimating large domain distributions that achieves optimal sample complexity, low communication, and fast computation using Hadamard matrices.

Contribution

The paper presents Hadamard Response, a simple, communication-efficient, and computationally fast local privatization scheme with optimal sample complexity for all privacy levels.

Findings

Achieves order optimal sample complexity for all ε

Requires only log k + 2 bits of communication per user

Runs approximately 100x faster than existing methods for k=10000

Abstract

We study the problem of estimating $k$-ary distributions under $\varepsilon$-local differential privacy. $n$ samples are distributed across users who send privatized versions of their sample to a central server. All previously known sample optimal algorithms require linear (in $k$) communication from each user in the high privacy regime $(\varepsilon=O(1))$, and run in time that grows as $n\cdot k$, which can be prohibitive for large domain size $k$. We propose Hadamard Response (HR}, a local privatization scheme that requires no shared randomness and is symmetric with respect to the users. Our scheme has order optimal sample complexity for all $\varepsilon$, a communication of at most $\log k+2$ bits per user, and nearly linear running time of $\tilde{O}(n + k)$. Our encoding and decoding are based on Hadamard matrices, and are simple to implement. The statistical performance relies on the coding theoretic aspects of Hadamard matrices, ie, the large Hamming distance between the rows. An efficient implementation of the algorithm using the Fast Walsh-Hadamard transform gives the computational gains. We compare our approach with Randomized Response (RR), RAPPOR, and subset-selection mechanisms (SS), both theoretically, and experimentally. For $k=10000$, our algorithm runs about 100x faster than SS, and RAPPOR.