Neural Machine Translation for Bilingually Scarce Scenarios: A Deep Multi-task Learning Approach

TL;DR

This paper introduces a multi-task learning approach that leverages monolingual source language resources to improve neural machine translation for low-resource language pairs, by incorporating auxiliary semantic and syntactic tasks.

Contribution

It presents a novel multi-task learning framework that uses auxiliary linguistic tasks to enhance translation quality in low-resource scenarios.

Findings

Improved translation quality for English-to-Farsi and English-to-Vietnamese.

Effective injection of semantic and syntactic knowledge into translation models.

Demonstrated benefits across three diverse language pairs.

Abstract

Neural machine translation requires large amounts of parallel training text to learn a reasonable-quality translation model. This is particularly inconvenient for language pairs for which enough parallel text is not available. In this paper, we use monolingual linguistic resources in the source side to address this challenging problem based on a multi-task learning approach. More specifically, we scaffold the machine translation task on auxiliary tasks including semantic parsing, syntactic parsing, and named-entity recognition. This effectively injects semantic and/or syntactic knowledge into the translation model, which would otherwise require a large amount of training bitext. We empirically evaluate and show the effectiveness of our multi-task learning approach on three translation tasks: English-to-French, English-to-Farsi, and English-to-Vietnamese.