Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systems

TL;DR

This paper introduces a novel method for deriving compact, low-dimensional contextual representations in Conformer-Transducer speech recognition systems, leading to improved accuracy by effectively utilizing cross-utterance context.

Contribution

It proposes a new attention pooling approach to learn efficient cross-utterance contextual features, outperforming previous methods in streaming speech recognition.

Findings

Achieved statistically significant WER reductions of up to 0.7% absolute.

Demonstrated improvements on the Gigaspeech corpus with 1000 hours of data.

Outperformed baseline models using internal context only.

Abstract

Current ASR systems are mainly trained and evaluated at the utterance level. Long range cross utterance context can be incorporated. A key task is to derive a suitable compact representation of the most relevant history contexts. In contrast to previous researches based on either LSTM-RNN encoded histories that attenuate the information from longer range contexts, or frame level concatenation of transformer context embeddings, in this paper compact low-dimensional cross utterance contextual features are learned in the Conformer-Transducer Encoder using specially designed attention pooling layers that are applied over efficiently cached preceding utterances history vectors. Experiments on the 1000-hr Gigaspeech corpus demonstrate that the proposed contextualized streaming Conformer-Transducers outperform the baseline using utterance internal context only with statistically significant WER reductions of 0.7% to 0.5% absolute (4.3% to 3.1% relative) on the dev and test data.