Exploring Parameter-Efficient Fine-Tuning and Backtranslation for the WMT 25 General Translation Task

TL;DR

This paper demonstrates that combining backtranslation and fine-tuning on small Japanese datasets significantly improves neural machine translation quality, outperforming individual techniques and offering an efficient approach for low-resource languages.

Contribution

It introduces a combined method of backtranslation and fine-tuning that enhances translation performance on low-resource Japanese datasets.

Findings

Backtranslation alone slightly improves COMET score from 0.460 to 0.468.

Fine-tuning on small datasets substantially increases COMET to 0.589.

Combining backtranslation and fine-tuning further boosts COMET to 0.597.

Abstract

In this paper, we explore the effectiveness of combining fine-tuning and backtranslation on a small Japanese corpus for neural machine translation. Starting from a baseline English{\textrightarrow}Japanese model (COMET = 0.460), we first apply backtranslation (BT) using synthetic data generated from monolingual Japanese corpora, yielding a modest increase (COMET = 0.468). Next, we fine-tune (FT) the model on a genuine small parallel dataset drawn from diverse Japanese news and literary corpora, achieving a substantial jump to COMET = 0.589 when using Mistral 7B. Finally, we integrate both backtranslation and fine-tuning{ -- }first augmenting the small dataset with BT generated examples, then adapting via FT{ -- }which further boosts performance to COMET = 0.597. These results demonstrate that, even with limited training data, the synergistic use of backtranslation and targeted fine-tuning on Japanese corpora can significantly enhance translation quality, outperforming each technique in isolation. This approach offers a lightweight yet powerful strategy for improving low-resource language pairs.