Extreme Adaptation for Personalized Neural Machine Translation

TL;DR

This paper introduces a simple, parameter-efficient method for personalizing neural machine translation by adapting output biases, leading to improved translation accuracy and better reflection of individual speaker traits.

Contribution

It proposes a novel bias adaptation technique for personalized NMT, which is simple, parameter-efficient, and effective across multiple languages.

Findings

Improved translation accuracy on TED talks datasets

Enhanced reflection of speaker traits in translations

Parameter-efficient adaptation method

Abstract

Every person speaks or writes their own flavor of their native language, influenced by a number of factors: the content they tend to talk about, their gender, their social status, or their geographical origin. When attempting to perform Machine Translation (MT), these variations have a significant effect on how the system should perform translation, but this is not captured well by standard one-size-fits-all models. In this paper, we propose a simple and parameter-efficient adaptation technique that only requires adapting the bias of the output softmax to each particular user of the MT system, either directly or through a factored approximation. Experiments on TED talks in three languages demonstrate improvements in translation accuracy, and better reflection of speaker traits in the target text.