Deep CLAS: Deep Contextual Listen, Attend and Spell

TL;DR

Deep CLAS enhances automatic speech recognition of rare words by integrating character-level encoding, bias loss, and enriched attention mechanisms, leading to significant improvements in named entity recognition accuracy.

Contribution

This work introduces deep CLAS, incorporating bias loss and character-level encoding with conformer models to better utilize contextual information in ASR.

Findings

65.78% relative recall increase in NER tasks

53.49% relative F1-score improvement

Effective use of character-level encoding and bias attention

Abstract

Contextual-LAS (CLAS) has been shown effective in improving Automatic Speech Recognition (ASR) of rare words. It relies on phrase-level contextual modeling and attention-based relevance scoring without explicit contextual constraint which lead to insufficient use of contextual information. In this work, we propose deep CLAS to use contextual information better. We introduce bias loss forcing model to focus on contextual information. The query of bias attention is also enriched to improve the accuracy of the bias attention score. To get fine-grained contextual information, we replace phrase-level encoding with character-level encoding and encode contextual information with conformer rather than LSTM. Moreover, we directly use the bias attention score to correct the output probability distribution of the model. Experiments using the public AISHELL-1 and AISHELL-NER. On AISHELL-1, compared to CLAS baselines, deep CLAS obtains a 65.78% relative recall and a 53.49% relative F1-score increase in the named entity recognition scene.