Assessing Vulnerabilities of Adversarial Learning Algorithm through Poisoning Attacks

TL;DR

This paper investigates the vulnerabilities of adversarial training (AT) to poisoning attacks, revealing that AT can be compromised through subtle data manipulations, which raises concerns for its use in security-sensitive AI applications.

Contribution

The study designs and tests novel clean-label poisoning attacks against AT, including targeted and untargeted methods, demonstrating AT's susceptibility to such malicious data manipulations.

Findings

AT can be effectively poisoned with minimal data modifications

Clean-label attacks can control model behavior on specific data points

Attacks can degrade overall model performance, even against robust training

Abstract

Adversarial training (AT) is a robust learning algorithm that can defend against adversarial attacks in the inference phase and mitigate the side effects of corrupted data in the training phase. As such, it has become an indispensable component of many artificial intelligence (AI) systems. However, in high-stake AI applications, it is crucial to understand AT's vulnerabilities to ensure reliable deployment. In this paper, we investigate AT's susceptibility to poisoning attacks, a type of malicious attack that manipulates training data to compromise the performance of the trained model. Previous work has focused on poisoning attacks against standard training, but little research has been done on their effectiveness against AT. To fill this gap, we design and test effective poisoning attacks against AT. Specifically, we investigate and design clean-label poisoning attacks, allowing attackers to imperceptibly modify a small fraction of training data to control the algorithm's behavior on a specific target data point. Additionally, we propose the clean-label untargeted attack, enabling attackers can attach tiny stickers on training data to degrade the algorithm's performance on all test data, where the stickers could serve as a signal against unauthorized data collection. Our experiments demonstrate that AT can still be poisoned, highlighting the need for caution when using vanilla AT algorithms in security-related applications. The code is at https://github.com/zjfheart/Poison-adv-training.git.