On Romanization for Model Transfer Between Scripts in Neural Machine Translation

TL;DR

This paper investigates the use of romanization to improve transfer learning in neural machine translation between languages with different scripts, highlighting its benefits, limitations, and the impact of different romanization tools.

Contribution

It introduces the exploration of romanization for model transfer in NMT, compares tools, and extends romanization to target-side translation with deromanization.

Findings

Romanization can improve transfer between related languages with different scripts.

Different romanization tools cause varying degrees of information loss.

Target-side romanization combined with deromanization can enhance translation quality.

Abstract

Transfer learning is a popular strategy to improve the quality of low-resource machine translation. For an optimal transfer of the embedding layer, the child and parent model should share a substantial part of the vocabulary. This is not the case when transferring to languages with a different script. We explore the benefit of romanization in this scenario. Our results show that romanization entails information loss and is thus not always superior to simpler vocabulary transfer methods, but can improve the transfer between related languages with different scripts. We compare two romanization tools and find that they exhibit different degrees of information loss, which affects translation quality. Finally, we extend romanization to the target side, showing that this can be a successful strategy when coupled with a simple deromanization model.