Improving Similar Language Translation With Transfer Learning

TL;DR

This paper explores transfer learning with pre-trained neural machine translation models to improve translation quality for low-resource similar language pairs, achieving top results in the WMT 2021 Shared Task.

Contribution

It demonstrates effective transfer learning techniques for low-resource similar languages, including novel submissions for French-Bambara, and achieves top BLEU scores in shared task evaluations.

Findings

Catalan-Spanish BLEU: 82.79

Portuguese-Spanish BLEU: 87.11

Top-ranked in WMT 2021 Shared Task

Abstract

We investigate transfer learning based on pre-trained neural machine translation models to translate between (low-resource) similar languages. This work is part of our contribution to the WMT 2021 Similar Languages Translation Shared Task where we submitted models for different language pairs, including French-Bambara, Spanish-Catalan, and Spanish-Portuguese in both directions. Our models for Catalan-Spanish ($82.79$ BLEU) and Portuguese-Spanish ($87.11$ BLEU) rank top 1 in the official shared task evaluation, and we are the only team to submit models for the French-Bambara pairs.