Mischief: A Simple Black-Box Attack Against Transformer Architectures

TL;DR

Mischief is a lightweight black-box attack method that generates realistic adversarial examples for transformer-based language models, significantly degrading their performance but also enabling improved robustness through retraining.

Contribution

We propose Mischief, a simple black-box adversarial attack method for language models that produces human-readable examples and demonstrates how retraining can restore or enhance model performance.

Findings

Adversarial samples reduce model accuracy by up to 20%.

Including adversarial examples in training restores baseline performance.

Training with Mischief examples can modestly improve model accuracy.

Abstract

We introduce Mischief, a simple and lightweight method to produce a class of human-readable, realistic adversarial examples for language models. We perform exhaustive experimentations of our algorithm on four transformer-based architectures, across a variety of downstream tasks, as well as under varying concentrations of said examples. Our findings show that the presence of Mischief-generated adversarial samples in the test set significantly degrades (by up to $20\%$) the performance of these models with respect to their reported baselines. Nonetheless, we also demonstrate that, by including similar examples in the training set, it is possible to restore the baseline scores on the adversarial test set. Moreover, for certain tasks, the models trained with Mischief set show a modest increase on performance with respect to their original, non-adversarial baseline.