The THUEE System Description for the IARPA OpenASR21 Challenge

TL;DR

This paper details the THUEE team's speech recognition system for the IARPA OpenASR21 Challenge, combining hybrid models with data augmentation and self-supervised learning to achieve top results.

Contribution

The paper introduces a hybrid ASR system with G2P lexicon extension and a self-supervised wav2vec2.0 approach, demonstrating effective integration for improved speech recognition.

Findings

G2P lexicon extension alleviates OOV issues.

Self-supervised wav2vec2.0 improves recognition performance.

Frontend feature extractor is crucial for wav2vec2.0 adaptation.

Abstract

This paper describes the THUEE team's speech recognition system for the IARPA Open Automatic Speech Recognition Challenge (OpenASR21), with further experiment explorations. We achieve outstanding results under both the Constrained and Constrained-plus training conditions. For the Constrained training condition, we construct our basic ASR system based on the standard hybrid architecture. To alleviate the Out-Of-Vocabulary (OOV) problem, we extend the pronunciation lexicon using Grapheme-to-Phoneme (G2P) techniques for both OOV and potential new words. Standard acoustic model structures such as CNN-TDNN-F and CNN-TDNN-F-A are adopted. In addition, multiple data augmentation techniques are applied. For the Constrained-plus training condition, we use the self-supervised learning framework wav2vec2.0. We experiment with various fine-tuning techniques with the Connectionist Temporal Classification (CTC) criterion on top of the publicly available pre-trained model XLSR-53. We find that the frontend feature extractor plays an important role when applying the wav2vec2.0 pre-trained model to the encoder-decoder based CTC/Attention ASR architecture. Extra improvements can be achieved by using the CTC model finetuned in the target language as the frontend feature extractor.