R-BI: Regularized Batched Inputs enhance Incremental Decoding Framework for Low-Latency Simultaneous Speech Translation

TL;DR

This paper introduces R-BI, a regularized batching method that improves incremental decoding for low-latency speech translation, reducing errors and achieving state-of-the-art results with minimal BLEU score loss.

Contribution

The paper proposes a novel Regularized Batched Inputs policy that enhances incremental decoding by increasing input diversity and regularization, applicable to both end-to-end and cascade systems.

Findings

Achieves low latency with less than 2 BLEU points loss.

Attains new state-of-the-art results on IWSLT SimulST tasks.

Effectively reduces output errors in incremental decoding.

Abstract

Incremental Decoding is an effective framework that enables the use of an offline model in a simultaneous setting without modifying the original model, making it suitable for Low-Latency Simultaneous Speech Translation. However, this framework may introduce errors when the system outputs from incomplete input. To reduce these output errors, several strategies such as Hold-$n$, LA-$n$, and SP-$n$ can be employed, but the hyper-parameter $n$ needs to be carefully selected for optimal performance. Moreover, these strategies are more suitable for end-to-end systems than cascade systems. In our paper, we propose a new adaptable and efficient policy named "Regularized Batched Inputs". Our method stands out by enhancing input diversity to mitigate output errors. We suggest particular regularization techniques for both end-to-end and cascade systems. We conducted experiments on IWSLT Simultaneous Speech Translation (SimulST) tasks, which demonstrate that our approach achieves low latency while maintaining no more than 2 BLEU points loss compared to offline systems. Furthermore, our SimulST systems attained several new state-of-the-art results in various language directions.