A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$\Delta$ Integration into Upcycled MoE

TL;DR

This paper introduces extmethod, a method to efficiently expand multilingual capabilities of large language models by integrating language-specific experts via parameter deltas, avoiding costly retraining.

Contribution

The paper proposes a novel approach to expand LLMs to new languages using MoE architecture and parameter deltas, bypassing extensive retraining and alignment.

Findings

extmethod improves performance on new languages while preserving original capabilities.

It outperforms baselines with similar FLOPs or parameters.

The approach is applicable across different models and post-training deltas.

Abstract

Expanding Large Language Models~(LLMs) to new languages is a costly endeavor, demanding extensive Continued Pre-Training~(CPT) and data-intensive alignment. While recent data-free merging techniques attempt to bypass alignment by fusing a multilingual CPT-enhanced model with its instruct counterpart, they are plagued by a critical trade-off: mitigating parameter conflicts to preserve original abilities inevitably dilutes new language acquisition, and vice-versa. To resolve this conflict, we introduce \method, which upcycles a dense model into a Mixture-of-Experts~(MoE) architecture, allocating different experts to different languages. Alignment ability is then transferred by grafting a MoE-expanded parameter delta~($\Delta_{\text{post}}$) to the CPT-enhanced base model, bypassing the complex alignment phase. Experiments demonstrate \method's superiority even against baselines with similar FLOPs or number of parameters; it improves performance on expanded languages while effectively preserving original capabilities. We further show our approach is highly applicable across different models and Post-training deltas.