Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models

TL;DR

This paper explores the impact of model and data scaling on multilingual machine translation using open large language models, introducing a new model that outperforms existing open and proprietary systems across 46 languages.

Contribution

It presents MiLMMT-46, a new open multilingual translation model that leverages model and data scaling, achieving state-of-the-art results across numerous languages.

Findings

MiLMMT-46 outperforms recent SOTA models in multilingual translation.

MiLMMT-46 achieves competitive results with proprietary systems.

Scaling models and data improves multilingual translation performance.

Abstract

Open large language models (LLMs) have demonstrated improving multilingual capabilities in recent years. In this paper, we present a study of open LLMs for multilingual machine translation (MT) across a range of languages, and investigate the effects of model scaling and data scaling when adapting open LLMs to multilingual MT through continual pretraining and instruction finetuning. Based on the Gemma3 model family, we develop MiLMMT-46, which achieves top-tier multilingual translation performance across 46 languages. Extensive experiments show that MiLMMT-46 consistently outperforms recent state-of-the-art (SOTA) models, including Seed-X, HY-MT-1.5, and TranslateGemma, and achieves competitive performance with strong proprietary systems such as Google Translate and Gemini 3 Pro. Models are released at https://huggingface.co/collections/xiaomi-research/milmmt-46. Codes are released at https://github.com/xiaomi-research/gemmax.