Lost In Translation: Generating Adversarial Examples Robust to Round-Trip Translation

TL;DR

This paper investigates how adversarial text examples are affected by round-trip translation and proposes using machine translation during generation to create more translation-robust adversarial attacks.

Contribution

It reveals that existing adversarial attacks lose effectiveness after translation and introduces a method integrating machine translation to improve robustness.

Findings

Existing attacks do not withstand round-trip translation.

Integrating machine translation enhances adversarial robustness.

Multilingual adversarial attack research is motivated.

Abstract

Language Models today provide a high accuracy across a large number of downstream tasks. However, they remain susceptible to adversarial attacks, particularly against those where the adversarial examples maintain considerable similarity to the original text. Given the multilingual nature of text, the effectiveness of adversarial examples across translations and how machine translations can improve the robustness of adversarial examples remain largely unexplored. In this paper, we present a comprehensive study on the robustness of current text adversarial attacks to round-trip translation. We demonstrate that 6 state-of-the-art text-based adversarial attacks do not maintain their efficacy after round-trip translation. Furthermore, we introduce an intervention-based solution to this problem, by integrating Machine Translation into the process of adversarial example generation and demonstrating increased robustness to round-trip translation. Our results indicate that finding adversarial examples robust to translation can help identify the insufficiency of language models that is common across languages, and motivate further research into multilingual adversarial attacks.