A Teacher-Student Framework for Zero-Resource Neural Machine Translation

TL;DR

This paper introduces a novel teacher-student framework for zero-resource neural machine translation, enabling training without direct parallel data by leveraging a pivot language and a teacher model.

Contribution

It proposes a new zero-resource NMT training method that uses a teacher-student approach guided by pivot-based models, addressing data scarcity in low-resource language pairs.

Findings

Achieved +3.0 BLEU points improvement over baseline pivot models

Effectively trains source-to-target NMT without parallel corpora

Demonstrated across multiple language pairs

Abstract

While end-to-end neural machine translation (NMT) has made remarkable progress recently, it still suffers from the data scarcity problem for low-resource language pairs and domains. In this paper, we propose a method for zero-resource NMT by assuming that parallel sentences have close probabilities of generating a sentence in a third language. Based on this assumption, our method is able to train a source-to-target NMT model ("student") without parallel corpora available, guided by an existing pivot-to-target NMT model ("teacher") on a source-pivot parallel corpus. Experimental results show that the proposed method significantly improves over a baseline pivot-based model by +3.0 BLEU points across various language pairs.