Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation

TL;DR

This paper investigates the phenomenon of language drift in multilingual retrieval-augmented generation models, revealing decoder-level causes and proposing a simple, model-agnostic decoding strategy to improve language consistency and task accuracy.

Contribution

It systematically characterizes language drift in multilingual RAG, identifies decoder collapse as the cause, and introduces Soft Constrained Decoding (SCD) as a novel, training-free mitigation method.

Findings

SCD improves language alignment across multiple datasets and languages.

Language drift is caused by decoder-level collapse, not comprehension failure.

English acts as a semantic attractor and fallback language in cross-lingual settings.

Abstract

Multilingual Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to perform knowledge-intensive tasks in multilingual settings by leveraging retrieved documents as external evidence. However, when the retrieved evidence differs in language from the user query and in-context exemplars, the model often exhibits language drift by generating responses in an unintended language. This phenomenon is especially pronounced during reasoning-intensive decoding, such as Chain-of-Thought (CoT) generation, where intermediate steps introduce further language instability. In this paper, we systematically study output language drift in multilingual RAG across multiple datasets, languages, and LLM backbones. Our controlled experiments reveal that the drift results not from comprehension failure but from decoder-level collapse, where dominant token distributions and high-frequency English patterns dominate the intended generation language. We further observe that English serves as a semantic attractor under cross-lingual conditions, emerging as both the strongest interference source and the most frequent fallback language. To mitigate this, we propose Soft Constrained Decoding (SCD), a lightweight, training-free decoding strategy that gently steers generation toward the target language by penalizing non-target-language tokens. SCD is model-agnostic and can be applied to any generation algorithm without modifying the architecture or requiring additional data. Experiments across three multilingual datasets and multiple typologically diverse languages show that SCD consistently improves language alignment and task performance, providing an effective and generalizable solution in multilingual RAG.