Personalization Strategies for End-to-End Speech Recognition Systems

TL;DR

This paper presents new biasing and rescoring techniques for end-to-end speech recognition systems that significantly improve personalized content recognition without sacrificing overall accuracy.

Contribution

It introduces a scalable word-level biasing algorithm and a novel second-pass de-biasing method that enhance personalized recognition in end-to-end systems.

Findings

Up to 16% improvement in personalized content recognition.

Additional 14% improvement with second-pass de-biasing.

Up to 2.5% accuracy gain for general use case.

Abstract

The recognition of personalized content, such as contact names, remains a challenging problem for end-to-end speech recognition systems. In this work, we demonstrate how first and second-pass rescoring strategies can be leveraged together to improve the recognition of such words. Following previous work, we use a shallow fusion approach to bias towards recognition of personalized content in the first-pass decoding. We show that such an approach can improve personalized content recognition by up to 16% with minimum degradation on the general use case. We describe a fast and scalable algorithm that enables our biasing models to remain at the word-level, while applying the biasing at the subword level. This has the advantage of not requiring the biasing models to be dependent on any subword symbol table. We also describe a novel second-pass de-biasing approach: used in conjunction with a first-pass shallow fusion that optimizes on oracle WER, we can achieve an additional 14% improvement on personalized content recognition, and even improve accuracy for the general use case by up to 2.5%.