Population Based Training for Data Augmentation and Regularization in Speech Recognition

TL;DR

This paper introduces population-based training to optimize data augmentation and regularization schedules in speech recognition, leading to significant performance improvements and reduced experimental effort.

Contribution

It demonstrates the effectiveness of population-based training for dynamic hyperparameter optimization in speech recognition, simplifying the process and improving accuracy.

Findings

8% relative WER improvement over baseline

Achieved 5.18% WER on LibriSpeech test-other

Effective optimization of SpecAugment and dropout schedules

Abstract

Varying data augmentation policies and regularization over the course of optimization has led to performance improvements over using fixed values. We show that population based training is a useful tool to continuously search those hyperparameters, within a fixed budget. This greatly simplifies the experimental burden and computational cost of finding such optimal schedules. We experiment in speech recognition by optimizing SpecAugment this way, as well as dropout. It compares favorably to a baseline that does not change those hyperparameters over the course of training, with an 8% relative WER improvement. We obtain 5.18% word error rate on LibriSpeech's test-other.