ACT-MNMT Auto-Constriction Turning for Multilingual Neural Machine Translation

TL;DR

This paper introduces ACT-MNMT, a supervised fine-tuning method that constructs constrained templates with trigger tokens to improve multilingual translation accuracy and reduce off-target issues in LLM-based models.

Contribution

It proposes a novel Auto-Constriction Turning mechanism that enhances multilingual NMT by automatically creating constrained templates with trigger tokens, orthogonal to prompt-based methods.

Findings

Significantly improves translation performance across multiple directions.

Reduces off-target translation phenomena.

Demonstrates effectiveness on WMT test sets.

Abstract

Large language model (LLM) has achieved promising performance in multilingual machine translation tasks through zero/few-shot prompts or prompt-tuning. However, due to the mixture of multilingual data during the pre-training of LLM, the LLM-based translation models face the off-target issue in both prompt-based methods, including a series of phenomena, namely instruction misunderstanding, translation with wrong language and over-generation. For this issue, this paper introduces an \textbf{\underline{A}}uto-\textbf{\underline{C}}onstriction \textbf{\underline{T}}urning mechanism for \textbf{\underline{M}}ultilingual \textbf{\underline{N}}eural \textbf{\underline{M}}achine \textbf{\underline{T}}ranslation (\model), which is a novel supervised fine-tuning mechanism and orthogonal to the traditional prompt-based methods. In this method, \model automatically constructs a constrained template in the target side by adding trigger tokens ahead of the ground truth. Furthermore, trigger tokens can be arranged and combined freely to represent different task semantics, and they can be iteratively updated to maximize the label likelihood. Experiments are performed on WMT test sets with multiple metrics, and the experimental results demonstrate that \model achieves substantially improved performance across multiple translation directions and reduce the off-target phenomena in the translation.