Can Domains Be Transferred Across Languages in Multi-Domain Multilingual Neural Machine Translation?

TL;DR

This paper explores whether domain information can be transferred across languages in multi-domain multilingual NMT, especially when in-domain data is missing, and demonstrates methods to improve zero-shot translation performance.

Contribution

It introduces the concept of multi-domain multilingual NMT and shows how to effectively transfer domain knowledge across languages, improving zero-shot translation and generalization.

Findings

MDML NMT boosts zero-shot BLEU scores by up to +10

Learning domain-aware representations enhances translation quality

Adding target-language tags to the encoder is effective

Abstract

Previous works mostly focus on either multilingual or multi-domain aspects of neural machine translation (NMT). This paper investigates whether the domain information can be transferred across languages on the composition of multi-domain and multilingual NMT, particularly for the incomplete data condition where in-domain bitext is missing for some language pairs. Our results in the curated leave-one-domain-out experiments show that multi-domain multilingual (MDML) NMT can boost zero-shot translation performance up to +10 gains on BLEU, as well as aid the generalisation of multi-domain NMT to the missing domain. We also explore strategies for effective integration of multilingual and multi-domain NMT, including language and domain tag combination and auxiliary task training. We find that learning domain-aware representations and adding target-language tags to the encoder leads to effective MDML-NMT.