Specializing Multi-domain NMT via Penalizing Low Mutual Information

TL;DR

This paper proposes a novel objective for multi-domain neural machine translation that penalizes low mutual information to enhance domain-specific learning, achieving state-of-the-art results.

Contribution

It introduces a mutual information-based penalty to improve domain specialization in multi-domain NMT models, which is a new approach in this area.

Findings

Achieved state-of-the-art performance on multi-domain NMT benchmarks.

Empirically demonstrated increased mutual information leads to better domain specialization.

The proposed method outperforms existing models in handling multiple domains.

Abstract

Multi-domain Neural Machine Translation (NMT) trains a single model with multiple domains. It is appealing because of its efficacy in handling multiple domains within one model. An ideal multi-domain NMT should learn distinctive domain characteristics simultaneously, however, grasping the domain peculiarity is a non-trivial task. In this paper, we investigate domain-specific information through the lens of mutual information (MI) and propose a new objective that penalizes low MI to become higher. Our method achieved the state-of-the-art performance among the current competitive multi-domain NMT models. Also, we empirically show our objective promotes low MI to be higher resulting in domain-specialized multi-domain NMT.