The NiuTrans Machine Translation Systems for WMT21

TL;DR

This paper details the NiuTrans neural machine translation systems for WMT21, employing advanced Transformer variants and techniques like back-translation and knowledge distillation to improve translation quality across multiple language pairs.

Contribution

Introduces NiuTrans systems using novel Transformer variants and multiple enhancement techniques for multilingual translation in WMT21.

Findings

Achieved competitive translation performance across 9 language directions.

Effectively utilized back-translation and knowledge distillation to boost model accuracy.

Demonstrated the effectiveness of Transformer-DLCL and ODE-Transformer architectures.

Abstract

This paper describes NiuTrans neural machine translation systems of the WMT 2021 news translation tasks. We made submissions to 9 language directions, including English$\leftrightarrow$$\{$Chinese, Japanese, Russian, Icelandic$\}$ and English$\rightarrow$Hausa tasks. Our primary systems are built on several effective variants of Transformer, e.g., Transformer-DLCL, ODE-Transformer. We also utilize back-translation, knowledge distillation, post-ensemble, and iterative fine-tuning techniques to enhance the model performance further.