CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation

TL;DR

This paper introduces CCL-XCoT, a two-stage fine-tuning method that significantly reduces hallucinations in multilingual language models by improving cross-lingual reasoning and semantic alignment, especially in low-resource languages.

Contribution

It proposes a novel curriculum-based contrastive learning framework combined with cross-lingual Chain-of-Thought prompting to mitigate hallucinations in multilingual models.

Findings

Reduces hallucination rates by up to 62%.

Enhances factual knowledge transfer across languages.

Does not rely on external retrieval or ensembles.

Abstract

Multilingual Large Language Models(MLLMs) demonstrate strong generalization across languages, yet they remain prone to hallucinations, especially in low-resource languages, due to training data imbalances. These hallucinations, which include inaccurate or fabricated outputs, are particularly problematic in domain-specific generation tasks (Chataigner et al., 2024). To address this challenge, we propose CCL-XCoT(Curriculum-based Contrastive Learning-based Cross-lingual Chain-of-Thought), a two-stage fine-tuning framework for mitigating hallucination in MLLMs. Our approach first enhances cross-lingual semantic alignment through curriculum-based contrastive learning combined with next-token prediction during continued pre-training. Building on this foundation, we then introduce a cross-lingual Chain-of-Thought (XCoT) prompting strategy during instruction fine-tuning, which guides the model to reason in a high-resource language before generating answers in the target low-resource language. Experimental results show that CCL-XCoT reduces hallucination rates by up to 62% and substantially improves factual knowledge transfer across language pairs, without relying on external retrieval or multi-model ensembles.