Low-Latency Neural Speech Translation

TL;DR

This paper presents a method to adapt neural machine translation systems for low-latency speech translation, effectively handling partial inputs and reducing correction steps without sacrificing translation quality.

Contribution

It introduces a novel adaptation approach using artificial data and multi-task learning to improve partial sentence translation in low-latency scenarios.

Findings

Reduced corrections by 45% during incremental translation

Maintained translation quality after adaptation

Effective adaptation without task-specific training data

Abstract

Through the development of neural machine translation, the quality of machine translation systems has been improved significantly. By exploiting advancements in deep learning, systems are now able to better approximate the complex mapping from source sentences to target sentences. But with this ability, new challenges also arise. An example is the translation of partial sentences in low-latency speech translation. Since the model has only seen complete sentences in training, it will always try to generate a complete sentence, though the input may only be a partial sentence. We show that NMT systems can be adapted to scenarios where no task-specific training data is available. Furthermore, this is possible without losing performance on the original training data. We achieve this by creating artificial data and by using multi-task learning. After adaptation, we are able to reduce the number of corrections displayed during incremental output construction by 45%, without a decrease in translation quality.