Investigation of Training Label Error Impact on RNN-T

TL;DR

This paper analyzes how different types of label errors affect RNN-T based speech recognition models, highlighting deletion errors as particularly harmful and emphasizing the importance of high-quality labels.

Contribution

It provides a quantitative analysis of label error impacts on RNN-T models and evaluates mitigation strategies, offering guidance for data pipeline design.

Findings

Deletion errors are more harmful than substitution and insertion errors.

Mitigation methods reduce but do not eliminate performance gaps caused by label errors.

High-quality labels remain crucial despite mitigation approaches.

Abstract

In this paper, we propose an approach to quantitatively analyze impacts of different training label errors to RNN-T based ASR models. The result shows deletion errors are more harmful than substitution and insertion label errors in RNN-T training data. We also examined label error impact mitigation approaches on RNN-T and found that, though all the methods mitigate the label-error-caused degradation to some extent, they could not remove the performance gap between the models trained with and without the presence of label errors. Based on the analysis results, we suggest to design data pipelines for RNN-T with higher priority on reducing deletion label errors. We also find that ensuring high-quality training labels remains important, despite of the existence of the label error mitigation approaches.