Improving Low-Resource Translation with Dictionary-Guided Fine-Tuning and RL: A Spanish-to-Wayuunaiki Study

TL;DR

This paper introduces a novel method combining dictionary-guided fine-tuning and reinforcement learning to improve low-resource translation, demonstrated on Spanish-Wayuunaiki with significant BLEU score gains.

Contribution

It presents a new approach integrating external dictionaries and reinforcement learning for low-resource language translation, specifically applied to Spanish-Wayuunaiki.

Findings

Achieved up to +3.37 BLEU improvement over previous methods.

Realized an 18% relative gain compared to supervised baseline without dictionary access.

Demonstrated the effectiveness of tool-augmented models with reinforcement learning.

Abstract

Low-resource machine translation remains a significant challenge for large language models (LLMs), which often lack exposure to these languages during pretraining and have limited parallel data for fine-tuning. We propose a novel approach that enhances translation for low-resource languages by integrating an external dictionary tool and training models end-to-end using reinforcement learning, in addition to supervised fine-tuning. Focusing on the Spanish-Wayuunaiki language pair, we frame translation as a tool-augmented decision-making problem in which the model can selectively consult a bilingual dictionary during generation. Our method combines supervised instruction tuning with Guided Reward Policy Optimization (GRPO), enabling the model to learn both when and how to use the tool effectively. BLEU similarity scores are used as rewards to guide this learning process. Preliminary results show that our tool-augmented models achieve up to +3.37 BLEU improvement over previous work, and a 18% relative gain compared to a supervised baseline without dictionary access, on the Spanish-Wayuunaiki test set from the AmericasNLP 2025 Shared Task. We also conduct ablation studies to assess the effects of model architecture and training strategy, comparing Qwen2.5-0.5B-Instruct with other models such as LLaMA and a prior NLLB-based system. These findings highlight the promise of combining LLMs with external tools and the role of reinforcement learning in improving translation quality in low-resource language settings.