Multilingual Non-Autoregressive Machine Translation without Knowledge Distillation

TL;DR

This paper introduces M-DAT, a non-autoregressive multilingual machine translation model that eliminates the need for knowledge distillation and improves generalization to unseen language pairs using pivot back-translation.

Contribution

It proposes M-DAT based on directed acyclic Transformer, removing the requirement for knowledge distillation in multilingual NMT.

Findings

Achieves state-of-the-art results in non-autoregressive MNMT

Eliminates the need for knowledge distillation

Improves generalization to unseen translation directions

Abstract

Multilingual neural machine translation (MNMT) aims at using one single model for multiple translation directions. Recent work applies non-autoregressive Transformers to improve the efficiency of MNMT, but requires expensive knowledge distillation (KD) processes. To this end, we propose an M-DAT approach to non-autoregressive multilingual machine translation. Our system leverages the recent advance of the directed acyclic Transformer (DAT), which does not require KD. We further propose a pivot back-translation (PivotBT) approach to improve the generalization to unseen translation directions. Experiments show that our M-DAT achieves state-of-the-art performance in non-autoregressive MNMT.