A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translation

TL;DR

This paper introduces a reinforcement learning-based interactive approach to neural machine translation that reduces human effort by learning from partial translation judgments, using uncertainty to trigger feedback, and updating models online.

Contribution

It proposes a novel interactive-predictive NMT method that leverages reinforcement learning, uncertainty-based feedback, and online adaptation to minimize human effort.

Findings

Reward signals improve translation quality metrics.

Human effort is reduced to about 5 feedback requests per input.

Online updates enable quick model adaptation.

Abstract

We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial translations. Secondly, human effort is further reduced by using the entropy of word predictions as uncertainty criterion to trigger feedback requests. Lastly, online updates of the model parameters after every interaction allow the model to adapt quickly. We show in simulation experiments that reward signals on partial translations significantly improve character F-score and BLEU compared to feedback on full translations only, while human effort can be reduced to an average number of $5$ feedback requests for every input.