Manipulation Attacks in Local Differential Privacy

TL;DR

This paper reveals that locally differentially private protocols are highly susceptible to adversarial manipulation, especially at high privacy levels or large input domains, risking significant distortion of data distributions.

Contribution

It systematically analyzes the vulnerability of non-interactive local differential privacy protocols to manipulation and compares the robustness of existing methods.

Findings

Adversaries controlling few users can obscure input distributions.

Vulnerability increases with higher privacy levels and larger input domains.

Existing protocols vary significantly in resistance to manipulation.

Abstract

Local differential privacy is a widely studied restriction on distributed algorithms that collect aggregates about sensitive user data, and is now deployed in several large systems. We initiate a systematic study of a fundamental limitation of locally differentially private protocols: they are highly vulnerable to adversarial manipulation. While any algorithm can be manipulated by adversaries who lie about their inputs, we show that any non-interactive locally differentially private protocol can be manipulated to a much greater extent. Namely, when the privacy level is high or the input domain is large, an attacker who controls a small fraction of the users in the protocol can completely obscure the distribution of the users' inputs. We also show that existing protocols differ greatly in their resistance to manipulation, even when they offer the same accuracy guarantee with honest execution. Our results suggest caution when deploying local differential privacy and reinforce the importance of efficient cryptographic techniques for emulating mechanisms from central differential privacy in distributed settings.