Information Loss in LLMs' Multilingual Translation: The Role of Training Data, Language Proximity, and Language Family

TL;DR

This paper investigates how training data, language proximity, and language family influence information loss in multilingual translation by LLMs, revealing that data volume and linguistic relationships significantly affect translation quality.

Contribution

It systematically analyzes the impact of linguistic factors and training data on translation quality in LLMs, highlighting the importance of language proximity and family.

Findings

Abundant training data mitigates linguistic divergence effects.

Languages closer to English yield higher translation quality.

Multiple distance metrics predict translation performance.

Abstract

Large language models have achieved impressive progress in multilingual translation, yet they continue to face challenges with certain language pairs-particularly those with limited training data or significant linguistic divergence from English. This study systematically investigates how training data, language proximity, and language family affect information loss in multilingual translation. We evaluate two large language models, GPT-4 and Llama 2, by performing round-trip translations. Translation quality was assessed using BLEU scores and BERT similarity metrics. Our results reveal a robust interaction between training data size and language distance: while abundant training data can mitigate the effects of linguistic divergence, languages structurally closer to English consistently yield higher translation quality in low-resource conditions. Among various distance metrics, orthographic, phylogenetic, syntactic, and geographical distances emerge as strong predictors of translation performance. Language family also exerts an independent influence. These findings contribute to a deeper understanding of the linguistic constraints shaping multilingual translation in large language models, emphasizing that translation quality is shaped not only by data volume but also by structural and typological relationships between languages.