Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation

TL;DR

This paper introduces Doubly Round-trip Translation (DRTT) to generate authentic adversarial examples for NMT, improving model robustness by better identifying adversarial samples and using them for training.

Contribution

It proposes DRTT as a new criterion for generating adversarial examples, addressing ambiguity in single RTT methods, and leverages bilingual adversarial pairs to enhance NMT robustness.

Findings

Significant robustness improvements on clean and noisy test sets.

Effective identification of authentic adversarial examples.

Enhanced NMT performance with DRTT-based training.

Abstract

Generating adversarial examples for Neural Machine Translation (NMT) with single Round-Trip Translation (RTT) has achieved promising results by releasing the meaning-preserving restriction. However, a potential pitfall for this approach is that we cannot decide whether the generated examples are adversarial to the target NMT model or the auxiliary backward one, as the reconstruction error through the RTT can be related to either. To remedy this problem, we propose a new criterion for NMT adversarial examples based on the Doubly Round-Trip Translation (DRTT). Specifically, apart from the source-target-source RTT, we also consider the target-source-target one, which is utilized to pick out the authentic adversarial examples for the target NMT model. Additionally, to enhance the robustness of the NMT model, we introduce the masked language models to construct bilingual adversarial pairs based on DRTT, which are used to train the NMT model directly. Extensive experiments on both the clean and noisy test sets (including the artificial and natural noise) show that our approach substantially improves the robustness of NMT models.