SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task

TL;DR

This paper describes SJTU-NICT's participation in WMT20, where they developed diverse neural machine translation systems for multiple language pairs and tracks, achieving top results in several directions.

Contribution

The paper introduces novel NMT techniques and data filtering strategies tailored for different language pairs and translation tracks, leading to state-of-the-art performance.

Findings

Won first place in English-Chinese, Polish-English, and German-Upper Sorbian translation tasks.

Implemented diverse NMT techniques including document-enhanced and pre-trained models.

Used TF-IDF filtering to improve domain relevance of training data.

Abstract

In this paper, we introduced our joint team SJTU-NICT 's participation in the WMT 2020 machine translation shared task. In this shared task, we participated in four translation directions of three language pairs: English-Chinese, English-Polish on supervised machine translation track, German-Upper Sorbian on low-resource and unsupervised machine translation tracks. Based on different conditions of language pairs, we have experimented with diverse neural machine translation (NMT) techniques: document-enhanced NMT, XLM pre-trained language model enhanced NMT, bidirectional translation as a pre-training, reference language based UNMT, data-dependent gaussian prior objective, and BT-BLEU collaborative filtering self-training. We also used the TF-IDF algorithm to filter the training set to obtain a domain more similar set with the test set for finetuning. In our submissions, the primary systems won the first place on English to Chinese, Polish to English, and German to Upper Sorbian translation directions.