Efficient Wait-k Models for Simultaneous Machine Translation

TL;DR

This paper explores efficient wait-k models for simultaneous machine translation, demonstrating their effectiveness across different architectures and latency levels, especially in low-resource spoken language settings.

Contribution

It introduces improved training methods for wait-k models using unidirectional encoders and multi-k training, and compares Transformer and 2D-convolutional architectures.

Findings

Wait-k models generalize well across latency levels.

2D-convolutional architecture is competitive with Transformers.

Models perform effectively in low-resource spoken language scenarios.

Abstract

Simultaneous machine translation consists in starting output generation before the entire input sequence is available. Wait-k decoders offer a simple but efficient approach for this problem. They first read k source tokens, after which they alternate between producing a target token and reading another source token. We investigate the behavior of wait-k decoding in low resource settings for spoken corpora using IWSLT datasets. We improve training of these models using unidirectional encoders, and training across multiple values of k. Experiments with Transformer and 2D-convolutional architectures show that our wait-k models generalize well across a wide range of latency levels. We also show that the 2D-convolution architecture is competitive with Transformers for simultaneous translation of spoken language.