From Brazilian Portuguese to European Portuguese

TL;DR

This paper develops and evaluates a neural translation system from Brazilian Portuguese to European Portuguese, addressing resource disparities and improving translation quality through fine-tuning large language models and human assessment.

Contribution

It introduces a new translation system specifically for Brazilian to European Portuguese, with a curated test set and comprehensive evaluation including human judgment.

Findings

Models outperform baseline translation methods.

Fine-tuning large language models improves translation accuracy.

ChatGPT 3.5 Turbo achieves the best results among tested models.

Abstract

Brazilian Portuguese and European Portuguese are two varieties of the same language and, despite their close similarities, they exhibit several differences. However, there is a significant disproportion in the availability of resources between the two variants, with Brazilian Portuguese having more abundant resources. This inequity can impact the quality of translation services accessible to European Portuguese speakers. To address this issue, we propose the development of a Brazilian Portuguese to European Portuguese translation system, leveraging recent advancements in neural architectures and models. To evaluate the performance of such systems, we manually curated a gold test set comprising 500 sentences across five different topics. Each sentence in the gold test set has two distinct references, facilitating a straightforward evaluation of future translation models. We experimented with various models by fine-tuning existing Large Language Models using parallel data extracted from movie subtitles and TED Talks transcripts in both Brazilian and European Portuguese. Our evaluation involved the use of conventional automatic metrics as well as a human evaluation. In addition, all models were compared against ChatGPT 3.5 Turbo, which currently yields the best results.