Scaling Low-Resource MT via Synthetic Data Generation with LLMs

TL;DR

This paper demonstrates that synthetic data generated by large language models can significantly enhance low-resource machine translation, especially when combined with effective training strategies and evaluated across diverse languages.

Contribution

The study introduces a method for generating high-quality synthetic parallel data using LLMs, extending to many language pairs, and provides practical insights into its application for low-resource MT.

Findings

Synthetic data improves low-resource MT performance

Effective training regimes enhance benefits of synthetic data

Public repository SynOPUS facilitates future research

Abstract

We investigate the potential of LLM-generated synthetic data for improving low-resource Machine Translation (MT). Focusing on seven diverse target languages, we construct a document-level synthetic corpus from English Europarl, and extend it via pivoting to 147 additional language pairs. Automatic and human evaluation confirm its overall high quality. We study its practical application by (i) identifying effective training regimes, (ii) comparing our data with the HPLT dataset, (iii) studying the effect of varying training data size, and (iiii) testing its utility beyond English-centric MT. Finally, we introduce SynOPUS, a public repository for synthetic parallel datasets. Our findings show that LLM-generated synthetic data, even when noisy, can substantially improve MT performance for low-resource languages.