On the Transferability of Adversarial Attacksagainst Neural Text Classifier

TL;DR

This paper systematically investigates how adversarial examples transfer across different neural text classifiers, analyzing factors affecting transferability and proposing methods to generate and diagnose such adversarial inputs.

Contribution

It is the first comprehensive study on transferability of adversarial text examples, introducing a genetic algorithm for ensemble attacks and deriving diagnostic word replacement rules.

Findings

Transferability varies with network architecture and tokenization.

Ensemble models can generate adversarial examples that fool most classifiers.

Adversarial examples reveal data biases and model vulnerabilities.

Abstract

Deep neural networks are vulnerable to adversarial attacks, where a small perturbation to an input alters the model prediction. In many cases, malicious inputs intentionally crafted for one model can fool another model. In this paper, we present the first study to systematically investigate the transferability of adversarial examples for text classification models and explore how various factors, including network architecture, tokenization scheme, word embedding, and model capacity, affect the transferability of adversarial examples. Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models. Such adversarial examples reflect the defects of the learning process and the data bias in the training set. Finally, we derive word replacement rules that can be used for model diagnostics from these adversarial examples.