Joint Dropout: Improving Generalizability in Low-Resource Neural Machine Translation through Phrase Pair Variables

TL;DR

This paper introduces Joint Dropout, a novel method that replaces phrases with variables in low-resource neural machine translation, significantly improving translation quality, robustness, and domain adaptability.

Contribution

It proposes a new phrase-variable substitution technique called Joint Dropout to enhance generalization in low-resource NMT systems.

Findings

Significant BLEU score improvements on low-resource language pairs

Enhanced robustness and domain adaptability of NMT models

Error analysis confirms improved compositionality and generalization

Abstract

Despite the tremendous success of Neural Machine Translation (NMT), its performance on low-resource language pairs still remains subpar, partly due to the limited ability to handle previously unseen inputs, i.e., generalization. In this paper, we propose a method called Joint Dropout, that addresses the challenge of low-resource neural machine translation by substituting phrases with variables, resulting in significant enhancement of compositionality, which is a key aspect of generalization. We observe a substantial improvement in translation quality for language pairs with minimal resources, as seen in BLEU and Direct Assessment scores. Furthermore, we conduct an error analysis, and find Joint Dropout to also enhance generalizability of low-resource NMT in terms of robustness and adaptability across different domains