Improving Noisy Student Training on Non-target Domain Data for Automatic Speech Recognition

TL;DR

This paper introduces a data filtering method called LM Filter to enhance Noisy Student Training for ASR, leading to significant CER reductions on non-target domain data and achieving state-of-the-art results without additional supervised data.

Contribution

The paper proposes a novel LM Filter strategy for data selection in NST, improving ASR performance on non-target domain data without extra supervised data.

Findings

Achieved 10.4% CER reduction over no-filter baseline.

Attained 3.31% CER on AISHELL-1 test set, the best without extra supervised data.

Reached 4.73% CER on AISHELL-2 with 1000 hours of supervised data.

Abstract

Noisy Student Training (NST) has recently demonstrated extremely strong performance in Automatic Speech Recognition(ASR). In this paper, we propose a data selection strategy named LM Filter to improve the performance of NST on non-target domain data in ASR tasks. Hypotheses with and without a Language Model are generated and the CER differences between them are utilized as a filter threshold. Results reveal that significant improvements of 10.4% compared with no data filtering baselines. We can achieve 3.31% CER in AISHELL-1 test set, which is best result from our knowledge without any other supervised data. We also perform evaluations on the supervised 1000 hour AISHELL-2 dataset and competitive results of 4.73% CER can be achieved.