Setting up the Data Printer with Improved English to Ukrainian Machine Translation
Yurii Paniv, Dmytro Chaplynskyi, Nikita Trynus, Volodymyr Kyrylov

TL;DR
This paper presents a method for building an improved English to Ukrainian translation system using supervised fine-tuning and data filtering, enabling faster dataset curation for Ukrainian language models.
Contribution
It introduces a novel training recipe combining noisy parallel data and quality filtering to enhance translation performance of a decoder-only model.
Findings
Dragoman outperforms previous state-of-the-art models on FLORES devtest.
The two-phase training improves translation quality.
Filtering with k-fold perplexity enhances data quality.
Abstract
To build large language models for Ukrainian we need to expand our corpora with large amounts of new algorithmic tasks expressed in natural language. Examples of task performance expressed in English are abundant, so with a high-quality translation system our community will be enabled to curate datasets faster. To aid this goal, we introduce a recipe to build a translation system using supervised finetuning of a large pretrained language model with a noisy parallel dataset of 3M pairs of Ukrainian and English sentences followed by a second phase of training using 17K examples selected by k-fold perplexity filtering on another dataset of higher quality. Our decoder-only model named Dragoman beats performance of previous state of the art encoder-decoder models on the FLORES devtest set.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLinguistics, Language Diversity, and Identity
