Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspective

TL;DR

This paper introduces a reference-free reinforcement learning fine-tuning method for encoder-decoder machine translation models, demonstrating consistent improvements across diverse languages without requiring parallel data.

Contribution

It applies Group Relative Policy Optimization with hybrid reference-free rewards to encoder-decoder MT models, showing systematic gains especially in low-resource scenarios.

Findings

Consistent improvements up to +5.03 chrF++ on Chinese languages.

Achieves competitive results without target-language data.

Gains are largest where baseline performance is weakest.

Abstract

Production machine translation relies overwhelmingly on encoder-decoder Seq2Seq models, yet reinforcement learning approaches to MT fine-tuning have largely targeted decoder-only LLMs at $\geq$7B parameters, with limited systematic study of encoder-decoder architectures. We apply Group Relative Policy Optimization to NLLB-200 (600M and 1.3B) using a hybrid reference-free reward (LaBSE and COMET-Kiwi) that requires no parallel data at fine-tuning time, evaluating across 13 typologically diverse languages. GRPO yields consistent improvements on all 13 languages, up to $+$5.03 chrF++ for Traditional Chinese, and, without any target-language data, competes with 3-epoch supervised fine-tuning on morphologically complex languages . We identify a consistent empirical pattern in which gains are largest where baseline performance is weakest and reward discriminability is highest, making this approach most effective precisely where parallel data is scarcest, and replicate this pattern across English and Spanish source languages.