Mitigating Data Poisoning Attacks to Local Differential Privacy

TL;DR

This paper presents a comprehensive framework to detect and mitigate data poisoning attacks in local differential privacy systems, improving data integrity and utility in adversarial environments.

Contribution

It introduces novel detection methods for malicious reports and hidden attack patterns, along with a new data recovery post-processing technique for LDP.

Findings

Detection methods require no extra data or attack info

Detection incurs minimal computational cost

Significant performance improvements over previous work

Abstract

The distributed nature of local differential privacy (LDP) invites data poisoning attacks and poses unforeseen threats to the underlying LDP-supported applications. In this paper, we propose a comprehensive mitigation framework for popular frequency estimation, which contains a suite of novel defenses, including malicious user detection, attack pattern recognition, and damaged utility recovery. In addition to existing attacks, we explore new adaptive adversarial activities for our mitigation design. For detection, we present a new method to precisely identify bogus reports and thus LDP aggregation can be performed over the ``clean'' data. When the attack behavior becomes stealthy and direct filtering out malicious users is difficult, we further propose a detection that can effectively recognize hidden adversarial patterns, thus facilitating the decision-making of service providers. These detection methods require no additional data and attack information and incur minimal computational cost. Our experiment demonstrates their excellent performance and substantial improvement over previous work in various settings. In addition, we conduct an empirical analysis of LDP post-processing for corrupted data recovery and propose a new post-processing method, through which we reveal new insights into protocol recommendations in practice and key design principles for future research.