Integration of Frame- and Label-synchronous Beam Search for Streaming Encoder-decoder Speech Recognition

TL;DR

This paper introduces a novel beam search method that combines frame- and label-synchronous decoding to improve streaming speech recognition accuracy and robustness, especially in out-of-domain scenarios.

Contribution

It proposes an integrated decoding scheme that leverages the strengths of both frame- and label-synchronous methods within a single beam search, enhancing performance and robustness.

Findings

Achieves lower error rates than existing methods.

Demonstrates robustness in out-of-domain conditions.

Effectively combines decoding strategies for improved accuracy.

Abstract

Although frame-based models, such as CTC and transducers, have an affinity for streaming automatic speech recognition, their decoding uses no future knowledge, which could lead to incorrect pruning. Conversely, label-based attention encoder-decoder mitigates this issue using soft attention to the input, while it tends to overestimate labels biased towards its training domain, unlike CTC. We exploit these complementary attributes and propose to integrate the frame- and label-synchronous (F-/L-Sync) decoding alternately performed within a single beam-search scheme. F-Sync decoding leads the decoding for block-wise processing, while L-Sync decoding provides the prioritized hypotheses using look-ahead future frames within a block. We maintain the hypotheses from both decoding methods to perform effective pruning. Experiments demonstrate that the proposed search algorithm achieves lower error rates compared to the other search methods, while being robust against out-of-domain situations.