ImportantAug: a data augmentation agent for speech

TL;DR

ImportantAug is a novel data augmentation technique for speech models that selectively adds noise to unimportant speech regions, improving recognition accuracy significantly over traditional methods.

Contribution

It introduces a learned importance prediction mechanism to guide targeted noise addition, enhancing speech recognition performance beyond conventional augmentation.

Findings

23.3% relative error reduction on GSC test set

25.4% error rate reduction compared to no augmentation

Outperforms traditional noise augmentation on noisy test sets

Abstract

We introduce ImportantAug, a technique to augment training data for speech classification and recognition models by adding noise to unimportant regions of the speech and not to important regions. Importance is predicted for each utterance by a data augmentation agent that is trained to maximize the amount of noise it adds while minimizing its impact on recognition performance. The effectiveness of our method is illustrated on version two of the Google Speech Commands (GSC) dataset. On the standard GSC test set, it achieves a 23.3% relative error rate reduction compared to conventional noise augmentation which applies noise to speech without regard to where it might be most effective. It also provides a 25.4% error rate reduction compared to a baseline without data augmentation. Additionally, the proposed ImportantAug outperforms the conventional noise augmentation and the baseline on two test sets with additional noise added.