Generating Natural Language Adversarial Examples through An Improved Beam Search Algorithm

TL;DR

This paper introduces an improved beam search algorithm for generating natural language adversarial examples, significantly increasing attack efficiency and success rate while reducing the number of queries needed.

Contribution

It proposes a novel attack model that outperforms benchmark methods in success rate and efficiency, with empirical validation on multiple models and datasets.

Findings

Achieves 100% attack success rate on BERT and BiLSTM with fewer queries

Reduces query count to 1/4 and 1/6.5 of state-of-the-art methods

Demonstrates good transferability of adversarial examples

Abstract

The research of adversarial attacks in the text domain attracts many interests in the last few years, and many methods with a high attack success rate have been proposed. However, these attack methods are inefficient as they require lots of queries for the victim model when crafting text adversarial examples. In this paper, a novel attack model is proposed, its attack success rate surpasses the benchmark attack methods, but more importantly, its attack efficiency is much higher than the benchmark attack methods. The novel method is empirically evaluated by attacking WordCNN, LSTM, BiLSTM, and BERT on four benchmark datasets. For instance, it achieves a 100\% attack success rate higher than the state-of-the-art method when attacking BERT and BiLSTM on IMDB, but the number of queries for the victim models only is 1/4 and 1/6.5 of the state-of-the-art method, respectively. Also, further experiments show the novel method has a good transferability on the generated adversarial examples.