Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework

TL;DR

This paper introduces a Length-Aware Framework for simultaneous machine translation that predicts full-sentence length and fills future source positions with positional encoding, effectively reducing position bias and improving translation quality.

Contribution

The paper proposes a novel Length-Aware Framework that bridges the gap between SiMT and full-sentence MT by predicting sentence length and filling future positions, reducing position bias.

Findings

Reduces position bias in SiMT models.

Improves translation performance on benchmark datasets.

Compatible with existing SiMT methods, including adaptive policies.

Abstract

Simultaneous machine translation (SiMT) starts translating while receiving the streaming source inputs, and hence the source sentence is always incomplete during translating. Different from the full-sentence MT using the conventional seq-to-seq architecture, SiMT often applies prefix-to-prefix architecture, which forces each target word to only align with a partial source prefix to adapt to the incomplete source in streaming inputs. However, the source words in the front positions are always illusoryly considered more important since they appear in more prefixes, resulting in position bias, which makes the model pay more attention on the front source positions in testing. In this paper, we first analyze the phenomenon of position bias in SiMT, and develop a Length-Aware Framework to reduce the position bias by bridging the structural gap between SiMT and full-sentence MT. Specifically, given the streaming inputs, we first predict the full-sentence length and then fill the future source position with positional encoding, thereby turning the streaming inputs into a pseudo full-sentence. The proposed framework can be integrated into most existing SiMT methods to further improve performance. Experiments on two representative SiMT methods, including the state-of-the-art adaptive policy, show that our method successfully reduces the position bias and thereby achieves better SiMT performance.