A Simple Baseline for Beam Search Reranking

TL;DR

This paper introduces a straightforward method for training rerankers to predict translation quality scores without extra data or parameters, providing a clean baseline for future research in beam search reranking.

Contribution

It presents a simple, effective baseline for reranking in machine translation that does not rely on large pretrained models or additional data.

Findings

The method effectively predicts BLEU scores for translation candidates.

It serves as a decoupled baseline for future beam search reranking research.

The approach improves the interpretability and comparability of reranking models.

Abstract

Reranking methods in machine translation aim to close the gap between common evaluation metrics (e.g. BLEU) and maximum likelihood learning and decoding algorithms. Prior works address this challenge by training models to rerank beam search candidates according to their predicted BLEU scores, building upon large models pretrained on massive monolingual corpora -- a privilege that was never made available to the baseline translation model. In this work, we examine a simple approach for training rerankers to predict translation candidates' BLEU scores without introducing additional data or parameters. Our approach can be used as a clean baseline, decoupled from external factors, for future research in this area.