Scheduled Multi-Task Learning: From Syntax to Translation

TL;DR

This paper introduces a scheduled multi-task learning framework that gradually shifts focus from syntax learning to translation, improving neural machine translation performance on large and low-resource datasets.

Contribution

It presents a novel training schedule that interleaves syntax and translation tasks, enhancing translation quality in neural encoder-decoder models.

Findings

Significant BLEU score improvements on WMT14 English-German translation.

Enhanced translation performance in low-resource settings.

Effective integration of syntax learning into translation models.

Abstract

Neural encoder-decoder models of machine translation have achieved impressive results, while learning linguistic knowledge of both the source and target languages in an implicit end-to-end manner. We propose a framework in which our model begins learning syntax and translation interleaved, gradually putting more focus on translation. Using this approach, we achieve considerable improvements in terms of BLEU score on relatively large parallel corpus (WMT14 English to German) and a low-resource (WIT German to English) setup.