Improving RNN-T ASR Performance with Date-Time and Location Awareness

TL;DR

This paper demonstrates that incorporating date-time and location context into RNN-T based ASR models significantly enhances speech recognition accuracy, especially with limited training data, by leveraging additional contextual information.

Contribution

The study introduces a method to integrate date-time and location context into RNN-T ASR models, showing notable performance improvements over baseline models.

Findings

Contextual signals improve ASR accuracy by up to 4.62%.

Performance gains are higher with smaller training datasets.

Context-aware models excel in specific domains without degrading overall performance.

Abstract

In this paper, we explore the benefits of incorporating context into a Recurrent Neural Network (RNN-T) based Automatic Speech Recognition (ASR) model to improve the speech recognition for virtual assistants. Specifically, we use meta information extracted from the time at which the utterance is spoken and the approximate location information to make ASR context aware. We show that these contextual information, when used individually, improves overall performance by as much as 3.48% relative to the baseline and when the contexts are combined, the model learns complementary features and the recognition improves by 4.62%. On specific domains, these contextual signals show improvements as high as 11.5%, without any significant degradation on others. We ran experiments with models trained on data of sizes 30K hours and 10K hours. We show that the scale of improvement with the 10K hours dataset is much higher than the one obtained with 30K hours dataset. Our results indicate that with limited data to train the ASR model, contextual signals can improve the performance significantly.