Quality-Aware Translation Tagging in Multilingual RAG system

TL;DR

This paper introduces QTT-RAG, a method that evaluates and tags translation quality in multilingual retrieval-augmented generation, improving factual accuracy and naturalness in low-resource language QA tasks.

Contribution

QTT-RAG explicitly assesses translation quality along three dimensions and uses this metadata to enhance multilingual response generation without content alteration.

Findings

QTT-RAG outperforms baseline models in low-resource language QA benchmarks.

The approach preserves factual integrity and translation reliability.

Effective across multiple languages and model sizes.

Abstract

Multilingual Retrieval-Augmented Generation (mRAG) often retrieves English documents and translates them into the query language for low-resource settings. However, poor translation quality degrades response generation performance. Existing approaches either assume sufficient translation quality or utilize the rewriting method, which introduces factual distortion and hallucinations. To mitigate these problems, we propose Quality-Aware Translation Tagging in mRAG (QTT-RAG), which explicitly evaluates translation quality along three dimensions-semantic equivalence, grammatical accuracy, and naturalness&fluency-and attach these scores as metadata without altering the original content. We evaluate QTT-RAG against CrossRAG and DKM-RAG as baselines in two open-domain QA benchmarks (XORQA, MKQA) using six instruction-tuned LLMs ranging from 2.4B to 14B parameters, covering two low-resource languages (Korean and Finnish) and one high-resource language (Chinese). QTT-RAG outperforms the baselines by preserving factual integrity while enabling generator models to make informed decisions based on translation reliability. This approach allows for effective usage of cross-lingual documents in low-resource settings with limited native language documents, offering a practical and robust solution across multilingual domains.