Improved Speech Pre-Training with Supervision-Enhanced Acoustic Unit

TL;DR

This paper introduces SeHuBERT, a speech pre-training method that enhances acoustic unit supervision to improve recognition accuracy and reduce training costs across multiple languages and datasets.

Contribution

The paper proposes a supervision-enhanced acoustic unit pattern for speech pre-training, significantly improving performance and efficiency over standard HuBERT.

Findings

Achieves 10.5% and 4.9% WER reduction on Turkmen speech recognition.

Reduces training cost by approximately 50% while maintaining performance.

Effective across multiple languages and data scales.

Abstract

Speech pre-training has shown great success in learning useful and general latent representations from large-scale unlabeled data. Based on a well-designed self-supervised learning pattern, pre-trained models can be used to serve lots of downstream speech tasks such as automatic speech recognition. In order to take full advantage of the labed data in low resource task, we present an improved pre-training method by introducing a supervision-enhanced acoustic unit (SEAU) pattern to intensify the expression of comtext information and ruduce the training cost. Encoder representations extracted from the SEAU pattern are used to generate more representative target units for HuBERT pre-training process. The proposed method, named SeHuBERT, achieves a relative word error rate reductions of 10.5% and 4.9% comared with the standard HuBERT on Turkmen speech recognition task with 500 hours and 100 hours fine-tuning data respectively. Extended to more languages and more data, SeHuBERT can aslo achieve a relative word error rate reductions of approximately 10% at half of the training cost compared with HuBERT.