Encouraging Neural Machine Translation to Satisfy Terminology Constraints

TL;DR

This paper introduces a training-based method for neural machine translation that ensures lexical constraints are satisfied without adding inference overhead, by augmenting data, masking constraint tokens, and biasing the loss function.

Contribution

The proposed approach is a novel training strategy that effectively encourages models to incorporate specified terminology constraints during translation.

Findings

Improves BLEU scores over baseline methods.

Increases the percentage of generated constraint terms.

Avoids additional inference complexity.

Abstract

We present a new approach to encourage neural machine translation to satisfy lexical constraints. Our method acts at the training step and thereby avoiding the introduction of any extra computational overhead at inference step. The proposed method combines three main ingredients. The first one consists in augmenting the training data to specify the constraints. Intuitively, this encourages the model to learn a copy behavior when it encounters constraint terms. Compared to previous work, we use a simplified augmentation strategy without source factors. The second ingredient is constraint token masking, which makes it even easier for the model to learn the copy behavior and generalize better. The third one, is a modification of the standard cross entropy loss to bias the model towards assigning high probabilities to constraint words. Empirical results show that our method improves upon related baselines in terms of both BLEU score and the percentage of generated constraint terms.