Left Behind: Cross-Lingual Transfer as a Bridge for Low-Resource Languages in Large Language Models

TL;DR

This study benchmarks large language models on low-resource languages, revealing systematic performance gaps and showing that cross-lingual transfer strategies have limited and architecture-dependent benefits.

Contribution

It provides a comprehensive evaluation of LLMs on low-resource languages and analyzes the effectiveness of cross-lingual transfer prompting strategies.

Findings

Performance gap of 13.8-16.7 percentage points between English and low-resource languages

Cross-lingual transfer prompting yields small gains (+2.2pp to +4.3pp) for bilingual models

Current LLMs underperform on low-resource languages, especially in accuracy

Abstract

We investigate how large language models perform on low-resource languages by benchmarking eight LLMs across five experimental conditions in English, Kazakh, and Mongolian. Using 50 hand-crafted questions spanning factual, reasoning, technical, and culturally grounded categories, we evaluate 2,000 responses on accuracy, fluency, and completeness. We find a consistent performance gap of 13.8-16.7 percentage points between English and low-resource language conditions, with models maintaining surface-level fluency while producing significantly less accurate content. Cross-lingual transfer-prompting models to reason in English before translating back-yields selective gains for bilingual architectures (+2.2pp to +4.3pp) but provides no benefit to English-dominant models. Our results demonstrate that current LLMs systematically underserve low-resource language communities, and that effective mitigation strategies are architecture-dependent rather than universal.