Improving Generalization of Deep Neural Network Acoustic Models with Length Perturbation and N-best Based Label Smoothing

TL;DR

This paper proposes length perturbation and n-best label smoothing techniques to enhance the generalization of deep neural network acoustic models in speech recognition, achieving state-of-the-art results on benchmark datasets.

Contribution

It introduces two novel data augmentation and label smoothing methods that improve DNN acoustic model performance in ASR tasks.

Findings

Both techniques individually improve model generalization.

Combining the techniques yields further performance gains.

Achieves state-of-the-art results on SWB300 dataset.

Abstract

We introduce two techniques, length perturbation and n-best based label smoothing, to improve generalization of deep neural network (DNN) acoustic models for automatic speech recognition (ASR). Length perturbation is a data augmentation algorithm that randomly drops and inserts frames of an utterance to alter the length of the speech feature sequence. N-best based label smoothing randomly injects noise to ground truth labels during training in order to avoid overfitting, where the noisy labels are generated from n-best hypotheses. We evaluate these two techniques extensively on the 300-hour Switchboard (SWB300) dataset and an in-house 500-hour Japanese (JPN500) dataset using recurrent neural network transducer (RNNT) acoustic models for ASR. We show that both techniques improve the generalization of RNNT models individually and they can also be complementary. In particular, they yield good improvements over a strong SWB300 baseline and give state-of-art performance on SWB300 using RNNT models.