CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation

TL;DR

CipherDAug introduces a ciphertext-based data augmentation method for neural machine translation using ROT-$k$ ciphered data, enhancing translation quality especially in low-resource scenarios without external data.

Contribution

It presents a novel ROT-$k$ ciphertext augmentation technique combined with multi-source training, improving NMT performance over existing methods.

Findings

Outperforms strong data augmentation techniques on multiple datasets.

Yields significant improvements in low-resource translation tasks.

Requires no external data beyond the original parallel corpus.

Abstract

We propose a novel data-augmentation technique for neural machine translation based on ROT-$k$ ciphertexts. ROT-$k$ is a simple letter substitution cipher that replaces a letter in the plaintext with the $k$th letter after it in the alphabet. We first generate multiple ROT-$k$ ciphertexts using different values of $k$ for the plaintext which is the source side of the parallel data. We then leverage this enciphered training data along with the original parallel data via multi-source training to improve neural machine translation. Our method, CipherDAug, uses a co-regularization-inspired training procedure, requires no external data sources other than the original training data, and uses a standard Transformer to outperform strong data augmentation techniques on several datasets by a significant margin. This technique combines easily with existing approaches to data augmentation, and yields particularly strong results in low-resource settings.