NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs

TL;DR

This paper introduces NiuTrans.LMT, a scalable multilingual translation model that addresses directional degeneration using strategic downsampling and parallel prompting, achieving competitive results across 60 languages.

Contribution

The paper proposes novel methods—strategic downsampling and parallel multilingual prompting—to improve multilingual translation quality and scalability in large language models.

Findings

NiuTrans.LMT models perform on par or better than larger baselines.

Strategic Downsampling mitigates directional degeneration in multilingual training.

Parallel Multilingual Prompting enhances cross-lingual transfer and test-time performance.

Abstract

Large language models have significantly advanced Multilingual Machine Translation (MMT), yet scaling to many languages while keeping quality robust across directions remains challenging. In this paper, we identify a failure mode of multilingual supervised fine-tuning (SFT) on multi-way parallel data: when such data are reused symmetrically around a pivot language (e.g., English), performance on reverse directions (X $\to$ pivot) can drop substantially. We term this phenomenon Directional Degeneration and attribute it to excessive many-to-one mappings, which encourage shortcut learning. We propose Strategic Downsampling (SD), a simple yet effective method to mitigate this degeneration. In addition, we introduce Parallel Multilingual Prompting (PMP), which augments translation instructions with an auxiliary parallel sentence to promote cross-lingual transfer during training and enables optional test-time enhancement when auxiliary translations are available. We further develop \textbf{NiuTrans.LMT} (\textbf{L}arge-scale \textbf{M}ultilingual \textbf{T}ranslation, abbreviated as \textbf{LMT}), a Chinese-English-centric suite of multilingual translation models spanning four sizes (0.6B/1.7B/4B/8B) and covering 60 languages and 234 directions. Comprehensive evaluations show that LMT is competitive among open-source MMT systems, and that our 4B LMT model performs on par with or better than substantially larger baselines. We release our models and project resources to support inclusive and scalable MMT.