Speculative Beam Search for Simultaneous Translation

TL;DR

This paper introduces a speculative beam search algorithm that enables beam search to be used effectively in simultaneous translation by hallucinating future steps, leading to significant improvements across multiple language pairs.

Contribution

It proposes a novel speculative beam search method that allows beam search to generate one word at a time in simultaneous translation, overcoming previous limitations.

Findings

Large improvements over previous methods in diverse language pairs

Enables beam search to be applied to single-word generation in real-time translation

Implicitly benefits from a target language model

Abstract

Beam search is universally used in full-sentence translation but its application to simultaneous translation remains non-trivial, where output words are committed on the fly. In particular, the recently proposed wait-k policy (Ma et al., 2019a) is a simple and effective method that (after an initial wait) commits one output word on receiving each input word, making beam search seemingly impossible. To address this challenge, we propose a speculative beam search algorithm that hallucinates several steps into the future in order to reach a more accurate decision, implicitly benefiting from a target language model. This makes beam search applicable for the first time to the generation of a single word in each step. Experiments over diverse language pairs show large improvements over previous work.