The Source-Target Domain Mismatch Problem in Machine Translation

TL;DR

This paper investigates how cultural and regional differences cause domain mismatch in low-resource machine translation, affecting training effectiveness, and proposes methods to mitigate this issue.

Contribution

It formalizes the source-target domain mismatch problem, introduces a metric to measure it, and empirically studies its impact on low-resource machine translation training.

Findings

Domain mismatch is more severe between languages from distant regions.

Back-translation performance degrades with increased domain mismatch.

Combining back-translation with self-training alleviates the negative effects.

Abstract

While we live in an increasingly interconnected world, different places still exhibit strikingly different cultures and many events we experience in our every day life pertain only to the specific place we live in. As a result, people often talk about different things in different parts of the world. In this work we study the effect of local context in machine translation and postulate that particularly in low resource settings this causes the domains of the source and target language to greatly mismatch, as the two languages are often spoken in further apart regions of the world with more distinctive cultural traits and unrelated local events. We first formalize the concept of source-target domain mismatch, propose a metric to quantify it, and provide empirical evidence corroborating our intuition that organic text produced by people speaking very different languages exhibits the most dramatic differences. We conclude with an empirical study of how source-target domain mismatch affects training of machine translation systems for low resource language pairs. In particular, we find that it severely affects back-translation, but the degradation can be alleviated by combining back-translation with self-training and by increasing the relative amount of target side monolingual data.