Zero-Resource Translation with Multi-Lingual Neural Machine Translation

TL;DR

This paper introduces a finetuning method for multi-lingual neural machine translation that enables effective zero-resource translation, matching or surpassing traditional models and pivot strategies with minimal additional parameters.

Contribution

It presents a novel finetuning algorithm combined with new translation strategies that significantly improve zero-resource translation performance in multi-lingual models.

Findings

Zero-resource translation performance matches single-pair models trained with 1M sentences.

Outperforms pivot-based translation strategies.

Requires only one additional copy of attention parameters.

Abstract

In this paper, we propose a novel finetuning algorithm for the recently introduced multi-way, mulitlingual neural machine translate that enables zero-resource machine translation. When used together with novel many-to-one translation strategies, we empirically show that this finetuning algorithm allows the multi-way, multilingual model to translate a zero-resource language pair (1) as well as a single-pair neural translation model trained with up to 1M direct parallel sentences of the same language pair and (2) better than pivot-based translation strategy, while keeping only one additional copy of attention-related parameters.