Enhancing Document-Level Machine Translation via Filtered Synthetic Corpora and Two-Stage LLM Adaptation

TL;DR

This paper introduces a two-stage fine-tuning approach for document-level machine translation using filtered synthetic data generated and refined by large language models, improving coherence and reducing hallucinations.

Contribution

It proposes a novel data augmentation and filtering pipeline combined with a two-stage fine-tuning strategy to enhance LLM-based document-level translation performance.

Findings

Improved translation coherence across documents.

Reduced hallucinations and omissions in generated translations.

Effective use of synthetic data with filtering metrics.

Abstract

In Machine Translation, Large Language Models (LLMs) have generally underperformed compared to conventional encoder-decoder systems and thus see limited adoption. However, LLMs excel at modeling contextual information, making them a natural fit for document-level translation tasks where coherence across sentences is crucial. Despite this potential, document-level MT with LLMs faces two key challenges: (1) the scarcity of large-scale, high-quality document-level parallel data; and (2) the propensity of LLMs to introduce hallucinations and omissions during generation. To address these challenges, we propose a two-stage fine-tuning strategy leveraging LLM-augmented document-level data. First, we augment data by converting summarization data into document-level parallel data using a LLM, and then filter it using multiple metrics, leveraging sacreBLEU, COMET, and LaBSE-based cosine similarity-to improve data quality. Finally, we employ a two-stage fine-tuning strategy: first fine-tuning on the abundant sentence-level MT resources, and then on the filtered document-level corpus.