Property Inference From Poisoning

TL;DR

This paper introduces a novel poisoning attack that increases information leakage in trained models, demonstrating high success rates with minimal data poisoning, thus highlighting a new security threat in sensitive applications.

Contribution

It presents the first property inference poisoning attack, theoretically proves its success under certain conditions, and empirically validates its effectiveness on real datasets.

Findings

Poisoning can significantly boost information leakage.

Attack achieves over 90% accuracy with 9-10% poisoning.

The attack is effective on real-world datasets.

Abstract

Property inference attacks consider an adversary who has access to the trained model and tries to extract some global statistics of the training data. In this work, we study property inference in scenarios where the adversary can maliciously control part of the training data (poisoning data) with the goal of increasing the leakage. Previous work on poisoning attacks focused on trying to decrease the accuracy of models either on the whole population or on specific sub-populations or instances. Here, for the first time, we study poisoning attacks where the goal of the adversary is to increase the information leakage of the model. Our findings suggest that poisoning attacks can boost the information leakage significantly and should be considered as a stronger threat model in sensitive applications where some of the data sources may be malicious. We describe our \emph{property inference poisoning attack} that allows the adversary to learn the prevalence in the training data of any property it chooses. We theoretically prove that our attack can always succeed as long as the learning algorithm used has good generalization properties. We then verify the effectiveness of our attack by experimentally evaluating it on two datasets: a Census dataset and the Enron email dataset. We were able to achieve above $90\%$ attack accuracy with $9-10\%$ poisoning in all of our experiments.