Data Poisoning against Differentially-Private Learners: Attacks and Defenses

TL;DR

This paper investigates the effectiveness of differential privacy as a defense against data poisoning attacks in machine learning, showing it provides resistance with limited attack scope but degrades with increased poisoning.

Contribution

It introduces attack algorithms targeting differentially-private learners and evaluates their effectiveness, highlighting the limits of privacy-based defenses against extensive data poisoning.

Findings

Differential privacy offers resistance to small-scale poisoning attacks.

Effectiveness of defenses diminishes as the attacker poisons more data.

Attack algorithms successfully compromise privacy-preserving learners with sufficient data poisoning.

Abstract

Data poisoning attacks aim to manipulate the model produced by a learning algorithm by adversarially modifying the training set. We consider differential privacy as a defensive measure against this type of attack. We show that such learners are resistant to data poisoning attacks when the adversary is only able to poison a small number of items. However, this protection degrades as the adversary poisons more data. To illustrate, we design attack algorithms targeting objective and output perturbation learners, two standard approaches to differentially-private machine learning. Experiments show that our methods are effective when the attacker is allowed to poison sufficiently many training items.