Contextual-Utterance Training for Automatic Speech Recognition

TL;DR

This paper introduces a novel contextual-utterance training method for streaming ASR systems that leverages past and future context to improve word error rate and latency, outperforming traditional RNN-T training.

Contribution

It proposes a dual-mode training approach that distills contextual knowledge from a teacher model, enhancing streaming ASR performance with better context utilization.

Findings

Reduced WER by over 6%

Lowered last token emission latency by more than 40ms

Outperformed classical RNN-T training methods

Abstract

Recent studies of streaming automatic speech recognition (ASR) recurrent neural network transducer (RNN-T)-based systems have fed the encoder with past contextual information in order to improve its word error rate (WER) performance. In this paper, we first propose a contextual-utterance training technique which makes use of the previous and future contextual utterances in order to do an implicit adaptation to the speaker, topic and acoustic environment. Also, we propose a dual-mode contextual-utterance training technique for streaming automatic speech recognition (ASR) systems. This proposed approach allows to make a better use of the available acoustic context in streaming models by distilling "in-place" the knowledge of a teacher, which is able to see both past and future contextual utterances, to the student which can only see the current and past contextual utterances. The experimental results show that a conformer-transducer system trained with the proposed techniques outperforms the same system trained with the classical RNN-T loss. Specifically, the proposed technique is able to reduce both the WER and the average last token emission latency by more than 6% and 40ms relative, respectively.