Reconstruction Attacks on Aggressive Relaxations of Differential Privacy

TL;DR

This paper demonstrates that relaxed versions of differential privacy, such as IDP and BDP, are vulnerable to reconstruction attacks that can recover significant dataset information despite claims of low privacy loss.

Contribution

It reveals the vulnerabilities of aggressive relaxations of differential privacy, showing they can be exploited to reconstruct datasets with low privacy loss.

Findings

IDP and BDP allow dataset reconstruction attacks.

Attacks succeed even with high-noise mechanisms claiming low privacy loss.

Reconstruction can yield more precise data than unprotected queries.

Abstract

Differential privacy is a widely accepted formal privacy definition that allows aggregate information about a dataset to be released while controlling privacy leakage for individuals whose records appear in the data. Due to the unavoidable tension between privacy and utility, there have been many works trying to relax the requirements of differential privacy to achieve greater utility. One class of relaxation, which is starting to gain support outside the privacy community is embodied by the definitions of individual differential privacy (IDP) and bootstrap differential privacy (BDP). The original version of differential privacy defines a set of neighboring database pairs and achieves its privacy guarantees by requiring that each pair of neighbors should be nearly indistinguishable to an attacker. The privacy definitions we study, however, aggressively reduce the set of neighboring pairs that are protected. Both IDP and BDP define a measure of "privacy loss" that satisfies formal privacy properties such as postprocessing invariance and composition, and achieve dramatically better utility than the traditional variants of differential privacy. However, there is a significant downside - we show that they allow a significant portion of the dataset to be reconstructed using algorithms that have arbitrarily low privacy loss under their privacy accounting rules. We demonstrate these attacks using the preferred mechanisms of these privacy definitions. In particular, we design a set of queries that, when protected by these mechanisms with high noise settings (i.e., with claims of very low privacy loss), yield more precise information about the dataset than if they were not protected at all.