Compensating for Data with Reasoning: Low-Resource Machine Translation with LLMs

TL;DR

This paper introduces novel prompting methods for low-resource machine translation using LLMs, demonstrating improved translation quality through syntactic coverage and reasoning capabilities, especially when parallel data is scarce.

Contribution

It proposes Fragment-Shot Prompting and Pivoted Fragment-Shot, novel techniques that enhance low-resource translation by segmenting input and leveraging reasoning abilities of LLMs.

Findings

Fragment-Shot Prompting improves translation quality for low-resource languages.

Stronger reasoning models better utilize retrieved knowledge for translation.

Prompt engineering has limited impact when translating from low-resource to high-resource languages.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in multilingual machine translation, sometimes even outperforming traditional neural systems. However, previous research has highlighted the challenges of using LLMs, particularly with prompt engineering, for low-resource languages. In this work, we introduce Fragment-Shot Prompting, a novel in-context learning method that segments input and retrieves translation examples based on syntactic coverage, along with Pivoted Fragment-Shot, an extension that enables translation without direct parallel data. We evaluate these methods using GPT-3.5, GPT-4o, o1-mini, LLaMA-3.3, and DeepSeek-R1 for translation between Italian and two Ladin variants, revealing three key findings: (1) Fragment-Shot Prompting is effective for translating into and between the studied low-resource languages, with syntactic coverage positively correlating with translation quality; (2) Models with stronger reasoning abilities make more effective use of retrieved knowledge, generally produce better translations, and enable Pivoted Fragment-Shot to significantly improve translation quality between the Ladin variants; and (3) prompt engineering offers limited, if any, improvements when translating from a low-resource to a high-resource language, where zero-shot prompting already yields satisfactory results. We publicly release our code and the retrieval corpora.