Effective and Imperceptible Adversarial Textual Attack via Multi-objectivization

TL;DR

This paper introduces HydraText, a multi-objective evolutionary algorithm that crafts adversarial text attacks balancing success and imperceptibility, outperforming existing methods in effectiveness and subtlety.

Contribution

It reformulates adversarial text attack as a multi-objective optimization problem and proposes HydraText, the first approach effective in both score-based and decision-based attack settings.

Findings

HydraText achieves high attack success rates.

AEs are more indistinguishable from human text.

Adversarial training with HydraText improves model robustness.

Abstract

The field of adversarial textual attack has significantly grown over the last few years, where the commonly considered objective is to craft adversarial examples (AEs) that can successfully fool the target model. However, the imperceptibility of attacks, which is also essential for practical attackers, is often left out by previous studies. In consequence, the crafted AEs tend to have obvious structural and semantic differences from the original human-written text, making them easily perceptible. In this work, we advocate leveraging multi-objectivization to address such issue. Specifically, we reformulate the problem of crafting AEs as a multi-objective optimization problem, where the attack imperceptibility is considered as an auxiliary objective. Then, we propose a simple yet effective evolutionary algorithm, dubbed HydraText, to solve this problem. To the best of our knowledge, HydraText is currently the only approach that can be effectively applied to both score-based and decision-based attack settings. Exhaustive experiments involving 44237 instances demonstrate that HydraText consistently achieves competitive attack success rates and better attack imperceptibility than the recently proposed attack approaches. A human evaluation study also shows that the AEs crafted by HydraText are more indistinguishable from human-written text. Finally, these AEs exhibit good transferability and can bring notable robustness improvement to the target model by adversarial training.