Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021

TL;DR

This paper presents fine-tuning strategies for low-resource language translation using pretrained Transformer models, achieving competitive BLEU scores for English-Marathi and English-Irish translation tasks.

Contribution

The paper introduces fine-tuning methods for pretrained multilingual Transformers to improve translation quality for low-resource languages.

Findings

Ranked 1st in English->Marathi translation

Ranked 1st in Irish->English translation

Ranked 2nd in English->Irish translation

Abstract

This paper reports the Machine Translation (MT) systems submitted by the IIITT team for the English->Marathi and English->Irish language pairs LoResMT 2021 shared task. The task focuses on getting exceptional translations for rather low-resourced languages like Irish and Marathi. We fine-tune IndicTrans, a pretrained multilingual NMT model for English->Marathi, using external parallel corpus as input for additional training. We have used a pretrained Helsinki-NLP Opus MT English->Irish model for the latter language pair. Our approaches yield relatively promising results on the BLEU metrics. Under the team name IIITT, our systems ranked 1, 1, and 2 in English->Marathi, Irish->English, and English->Irish, respectively.