Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognition

TL;DR

This paper introduces a decoder-agnostic, end-to-end neural associative memory approach for fast on-device personalized speech recognition, significantly improving recognition accuracy over traditional re-scoring methods.

Contribution

The work presents a novel neural associative memory model enabling rapid, decoder-agnostic contextual adaptation for on-device speech recognition personalization.

Findings

12% relative WER reduction

15.7% entity mention F1-score improvement

Effective on-device personalization simulation

Abstract

Fast contextual adaptation has shown to be effective in improving Automatic Speech Recognition (ASR) of rare words and when combined with an on-device personalized training, it can yield an even better recognition result. However, the traditional re-scoring approaches based on an external language model is prone to diverge during the personalized training. In this work, we introduce a model-based end-to-end contextual adaptation approach that is decoder-agnostic and amenable to on-device personalization. Our on-device simulation experiments demonstrate that the proposed approach outperforms the traditional re-scoring technique by 12% relative WER and 15.7% entity mention specific F1-score in a continues personalization scenario.