Enriching Under-Represented Named-Entities To Improve Speech Recognition Performance

TL;DR

This paper introduces methods to enrich under-represented named-entities in speech recognition systems by augmenting training data, improving language models, and rescoring lattices, leading to better recognition of rare entities.

Contribution

It proposes a comprehensive approach combining exemplar utterances, enriched embeddings, and lattice rescoring to enhance recognition of under-represented named-entities in ASR.

Findings

Improved UR-NE occurrence in word lattices.

Enhanced recognition accuracy for under-represented entities.

Effective boosting of likelihood scores for UR-NEs.

Abstract

Automatic speech recognition (ASR) for under-represented named-entity (UR-NE) is challenging due to such named-entities (NE) have insufficient instances and poor contextual coverage in the training data to learn reliable estimates and representations. In this paper, we propose approaches to enriching UR-NEs to improve speech recognition performance. Specifically, our first priority is to ensure those UR-NEs to appear in the word lattice if there is any. To this end, we make exemplar utterances for those UR-NEs according to their categories (e.g. location, person, organization, etc.), ending up with an improved language model (LM) that boosts the UR-NE occurrence in the word lattice. With more UR-NEs appearing in the lattice, we then boost the recognition performance through lattice rescoring methods. We first enrich the representations of UR-NEs in a pre-trained recurrent neural network LM (RNNLM) by borrowing the embedding representations of the rich-represented NEs (RR-NEs), yielding the lattices that statistically favor the UR-NEs. Finally, we directly boost the likelihood scores of the utterances containing UR-NEs and gain further performance improvement.