Unsupervised Domain Adaptation for Neural Machine Translation with Domain-Aware Feature Embeddings

TL;DR

This paper introduces a novel unsupervised domain adaptation method for neural machine translation that uses domain-aware feature embeddings, improving translation quality across different domains, especially when domain-specific data is limited.

Contribution

The paper proposes a new approach using domain-aware feature embeddings learned via an auxiliary language modeling task for better domain adaptation in NMT.

Findings

Consistent improvements in translation quality across multiple settings.

Combining domain-aware embeddings with back translation yields further gains.

Effective control of domain-specific outputs in neural machine translation.

Abstract

The recent success of neural machine translation models relies on the availability of high quality, in-domain data. Domain adaptation is required when domain-specific data is scarce or nonexistent. Previous unsupervised domain adaptation strategies include training the model with in-domain copied monolingual or back-translated data. However, these methods use generic representations for text regardless of domain shift, which makes it infeasible for translation models to control outputs conditional on a specific domain. In this work, we propose an approach that adapts models with domain-aware feature embeddings, which are learned via an auxiliary language modeling task. Our approach allows the model to assign domain-specific representations to words and output sentences in the desired domain. Our empirical results demonstrate the effectiveness of the proposed strategy, achieving consistent improvements in multiple experimental settings. In addition, we show that combining our method with back translation can further improve the performance of the model.