Neural machine translation system for Lezgian, Russian and Azerbaijani languages

TL;DR

This paper introduces the first neural machine translation system for Lezgian, Azerbaijani, and Russian, including datasets and experiments that evaluate translation quality and zero-shot capabilities.

Contribution

It provides the first neural translation model and datasets for Lezgian, along with analysis of training data effects and zero-shot translation performance.

Findings

Achieved BLEU scores up to 29.48 for Lezgian-Russian translation.

Zero-shot translation shows high fluency but frequent refusal to translate.

Contributed datasets and sentence encoder for Lezgian language.

Abstract

We release the first neural machine translation system for translation between Russian, Azerbaijani and the endangered Lezgian languages, as well as monolingual and parallel datasets collected and aligned for training and evaluating the system. Multiple experiments are conducted to identify how different sets of training language pairs and data domains can influence the resulting translation quality. We achieve BLEU scores of 26.14 for Lezgian-Azerbaijani, 22.89 for Azerbaijani-Lezgian, 29.48 for Lezgian-Russian and 24.25 for Russian-Lezgian pairs. The quality of zero-shot translation is assessed on a Large Language Model, showing its high level of fluency in Lezgian. However, the model often refuses to translate, justifying itself with its incompetence. We contribute our translation model along with the collected parallel and monolingual corpora and sentence encoder for the Lezgian language.