When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation?

TL;DR

This paper introduces a contrastive learning approach at the word level to improve many-to-many neural machine translation, achieving notable BLEU score gains without relying on high-quality bilingual dictionaries.

Contribution

It proposes a novel word-level contrastive objective that leverages automatically learned word alignments to enhance many-to-many NMT performance.

Findings

Achieves 0.8 BLEU score improvements across several language pairs.

Encoder's sentence retrieval performance correlates with translation quality.

Highlights the importance of encoder's sentence retrieval in translation accuracy.

Abstract

Word alignment has proven to benefit many-to-many neural machine translation (NMT). However, high-quality ground-truth bilingual dictionaries were used for pre-editing in previous methods, which are unavailable for most language pairs. Meanwhile, the contrastive objective can implicitly utilize automatically learned word alignment, which has not been explored in many-to-many NMT. This work proposes a word-level contrastive objective to leverage word alignments for many-to-many NMT. Empirical results show that this leads to 0.8 BLEU gains for several language pairs. Analyses reveal that in many-to-many NMT, the encoder's sentence retrieval performance highly correlates with the translation quality, which explains when the proposed method impacts translation. This motivates future exploration for many-to-many NMT to improve the encoder's sentence retrieval performance.