Dynamic Data Selection for Neural Machine Translation

TL;DR

This paper investigates the effectiveness of data selection techniques for neural machine translation and introduces a dynamic data selection method called gradual fine-tuning, which improves translation quality.

Contribution

The paper demonstrates that dynamic data selection, specifically gradual fine-tuning, enhances NMT performance beyond traditional static data selection methods.

Findings

Gradual fine-tuning improves BLEU scores by up to +3.1 over baseline.

Data selection benefits are less pronounced in NMT compared to PBMT.

Dynamic data selection outperforms static approaches in NMT training.

Abstract

Intelligent selection of training data has proven a successful technique to simultaneously increase training efficiency and translation performance for phrase-based machine translation (PBMT). With the recent increase in popularity of neural machine translation (NMT), we explore in this paper to what extent and how NMT can also benefit from data selection. While state-of-the-art data selection (Axelrod et al., 2011) consistently performs well for PBMT, we show that gains are substantially lower for NMT. Next, we introduce dynamic data selection for NMT, a method in which we vary the selected subset of training data between different training epochs. Our experiments show that the best results are achieved when applying a technique we call gradual fine-tuning, with improvements up to +2.6 BLEU over the original data selection approach and up to +3.1 BLEU over a general baseline.