Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesis

TL;DR

This paper proposes an unsupervised data augmentation method using text-to-speech synthesis to improve accented speech recognition, achieving significant word error rate reductions in experiments with Wav2vec2.0 models.

Contribution

It introduces a novel unsupervised TTS-based data augmentation approach for accented speech recognition, reducing reliance on manual transcriptions.

Findings

Up to 6.1% relative WER reduction with synthetic accented speech data.

Effective use of small accented speech datasets for TTS training.

Improved ASR performance in accented speech scenarios.

Abstract

This paper investigates the use of unsupervised text-to-speech synthesis (TTS) as a data augmentation method to improve accented speech recognition. TTS systems are trained with a small amount of accented speech training data and their pseudo-labels rather than manual transcriptions, and hence unsupervised. This approach enables the use of accented speech data without manual transcriptions to perform data augmentation for accented speech recognition. Synthetic accented speech data, generated from text prompts by using the TTS systems, are then combined with available non-accented speech data to train automatic speech recognition (ASR) systems. ASR experiments are performed in a self-supervised learning framework using a Wav2vec2.0 model which was pre-trained on large amount of unsupervised accented speech data. The accented speech data for training the unsupervised TTS are read speech, selected from L2-ARCTIC and British Isles corpora, while spontaneous conversational speech from the Edinburgh international accents of English corpus are used as the evaluation data. Experimental results show that Wav2vec2.0 models which are fine-tuned to downstream ASR task with synthetic accented speech data, generated by the unsupervised TTS, yield up to 6.1% relative word error rate reductions compared to a Wav2vec2.0 baseline which is fine-tuned with the non-accented speech data from Librispeech corpus.