GRRM: Group Relative Reward Modeling for Machine Translation

TL;DR

This paper introduces GRRM, a novel group-based reward model for machine translation that jointly evaluates candidate groups to improve ranking accuracy and translation quality.

Contribution

The paper proposes the Group Relative Reward Model (GRRM), a new approach that processes candidate groups jointly for better intra-group ranking in machine translation.

Findings

GRRM achieves competitive ranking accuracy.

Integrating GRRM improves translation quality.

Framework enhances reasoning capabilities in translation.

Abstract

While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Machine Translation hinges on accurate intra-group ranking. We identify that standard Scalar Quality Metrics (SQM) fall short in this context; by evaluating candidates in isolation, they lack the comparative context necessary to distinguish fine-grained linguistic nuances. To address this, we introduce the Group Quality Metric (GQM) paradigm and instantiate it via the Group Relative Reward Model (GRRM). Unlike traditional independent scorers, GRRM processes the entire candidate group jointly, leveraging comparative analysis to rigorously resolve relative quality and adaptive granularity. Empirical evaluations confirm that GRRM achieves competitive ranking accuracy among all baselines. Building on this foundation, we integrate GRRM into the GRPO training loop to optimize the translation policy. Experimental results demonstrate that our framework not only improves general translation quality but also unlocks reasoning capabilities comparable to state-of-the-art reasoning models. We release codes, datasets, and model checkpoints at https://github.com/NJUNLP/GRRM.