MultiSlav: Using Cross-Lingual Knowledge Transfer to Combat the Curse of Multilinguality

TL;DR

This paper investigates cross-lingual transfer in multilingual NMT for Slavic languages, demonstrating benefits in low-resource and zero-shot translation, and provides open-source models to advance research in under-studied language families.

Contribution

It introduces novel approaches for leveraging cross-lingual knowledge transfer in Slavic NMT and releases state-of-the-art models for these languages.

Findings

Cross-lingual transfer improves low-resource translation quality.

Zero-shot translation benefits are demonstrated in Slavic languages.

Open-source models are provided for community use.

Abstract

Does multilingual Neural Machine Translation (NMT) lead to The Curse of the Multlinguality or provides the Cross-lingual Knowledge Transfer within a language family? In this study, we explore multiple approaches for extending the available data-regime in NMT and we prove cross-lingual benefits even in 0-shot translation regime for low-resource languages. With this paper, we provide state-of-the-art open-source NMT models for translating between selected Slavic languages. We released our models on the HuggingFace Hub (https://hf.co/collections/allegro/multislav-6793d6b6419e5963e759a683) under the CC BY 4.0 license. Slavic language family comprises morphologically rich Central and Eastern European languages. Although counting hundreds of millions of native speakers, Slavic Neural Machine Translation is under-studied in our opinion. Recently, most NMT research focuses either on: high-resource languages like English, Spanish, and German - in WMT23 General Translation Task 7 out of 8 task directions are from or to English; massively multilingual models covering multiple language groups; or evaluation techniques.