Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM

TL;DR

This paper introduces Mufu, a multilingual fused learning approach that enhances low-resource translation by converting translation into a post-editing task using auxiliary candidates, improving performance over existing models.

Contribution

Mufu leverages auxiliary translation candidates and instruction prompts to improve low-resource language translation with LLMs, demonstrating robustness and efficiency gains.

Findings

Outperforms NLLB 1.3B in 64% of low-resource pairs

Models fine-tuned with Mufu are robust to auxiliary candidate quality

Distilled models maintain 3.1 chrF improvement over baselines

Abstract

Multilingual large language models (LLMs) are great translators, but this is largely limited to high-resource languages. For many LLMs, translating in and out of low-resource languages remains a challenging task. To maximize data efficiency in this low-resource setting, we introduce Mufu, which includes a selection of automatically generated multilingual candidates and an instruction to correct inaccurate translations in the prompt. Mufu prompts turn a translation task into a postediting one, and seek to harness the LLM's reasoning capability with auxiliary translation candidates, from which the model is required to assess the input quality, align the semantics cross-lingually, copy from relevant inputs and override instances that are incorrect. Our experiments on En-XX translations over the Flores-200 dataset show LLMs finetuned against Mufu-style prompts are robust to poor quality auxiliary translation candidates, achieving performance superior to NLLB 1.3B distilled model in 64% of low- and very-low-resource language pairs. We then distill these models to reduce inference cost, while maintaining on average 3.1 chrF improvement over finetune-only baseline in low-resource translations.