Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints

TL;DR

This paper introduces Aligned Constrained Training (ACT), a plug-in algorithm that enhances non-autoregressive translation models by better handling low-frequency lexical constraints, improving translation quality and constraint preservation.

Contribution

It proposes a novel plug-in method, ACT, that improves constrained NAT models' ability to handle rare lexical constraints by leveraging source-side context.

Findings

Improves constraint preservation in NAT models.

Enhances translation quality for rare constraints.

Outperforms baseline constrained NAT models.

Abstract

However, current autoregressive approaches suffer from high latency. In this paper, we focus on non-autoregressive translation (NAT) for this problem for its efficiency advantage. We identify that current constrained NAT models, which are based on iterative editing, do not handle low-frequency constraints well. To this end, we propose a plug-in algorithm for this line of work, i.e., Aligned Constrained Training (ACT), which alleviates this problem by familiarizing the model with the source-side context of the constraints. Experiments on the general and domain datasets show that our model improves over the backbone constrained NAT model in constraint preservation and translation quality, especially for rare constraints.