Pronunciation-aware unique character encoding for RNN Transducer-based Mandarin speech recognition

TL;DR

This paper introduces a pronunciation-aware unique character encoding for Mandarin RNN Transducer ASR, effectively addressing homophone issues and improving recognition accuracy by combining pronunciation and character indices.

Contribution

It proposes a novel encoding method that integrates pronunciation and character indices, enabling homophone resolution in Mandarin end-to-end ASR systems.

Findings

Improved recognition accuracy on Aishell and MagicData datasets.

Effective homophone problem mitigation through the new encoding.

One-to-one mapping simplifies decoding process.

Abstract

For Mandarin end-to-end (E2E) automatic speech recognition (ASR) tasks, compared to character-based modeling units, pronunciation-based modeling units could improve the sharing of modeling units in model training but meet homophone problems. In this study, we propose to use a novel pronunciation-aware unique character encoding for building E2E RNN-T-based Mandarin ASR systems. The proposed encoding is a combination of pronunciation-base syllable and character index (CI). By introducing the CI, the RNN-T model can overcome the homophone problem while utilizing the pronunciation information for extracting modeling units. With the proposed encoding, the model outputs can be converted into the final recognition result through a one-to-one mapping. We conducted experiments on Aishell and MagicData datasets, and the experimental results showed the effectiveness of the proposed method.