Single-Queue Decoding for Neural Machine Translation

TL;DR

This paper introduces a single-queue decoding algorithm for neural machine translation that improves hypothesis selection by revisiting discarded hypotheses, leading to better translation quality.

Contribution

It proposes a novel decoding method using a single priority queue and a penalty function, enhancing hypothesis exploration over traditional beam search.

Findings

Improved translation quality over beam search.

Better hypothesis management with revisiting discarded hypotheses.

Effective length penalty improves translation accuracy.

Abstract

Neural machine translation models rely on the beam search algorithm for decoding. In practice, we found that the quality of hypotheses in the search space is negatively affected owing to the fixed beam size. To mitigate this problem, we store all hypotheses in a single priority queue and use a universal score function for hypothesis selection. The proposed algorithm is more flexible as the discarded hypotheses can be revisited in a later step. We further design a penalty function to punish the hypotheses that tend to produce a final translation that is much longer or shorter than expected. Despite its simplicity, we show that the proposed decoding algorithm is able to select hypotheses with better qualities and improve the translation performance.