Exploring Speech Enhancement for Low-resource Speech Synthesis

TL;DR

This paper investigates how speech enhancement models can improve low-resource speech synthesis by augmenting training data, demonstrating significant quality improvements in Arabic TTS systems and analyzing the impact of speech distortion.

Contribution

It introduces a pipeline applying TF-GridNet speech enhancement to low-resource datasets for TTS training and provides empirical analysis of its effects on performance.

Findings

Enhanced TTS performance on low-resource Arabic datasets.

Speech enhancement improves ASR WER metrics.

Analysis of speech distortion effects on TTS quality.

Abstract

High-quality and intelligible speech is essential to text-to-speech (TTS) model training, however, obtaining high-quality data for low-resource languages is challenging and expensive. Applying speech enhancement on Automatic Speech Recognition (ASR) corpus mitigates the issue by augmenting the training data, while how the nonlinear speech distortion brought by speech enhancement models affects TTS training still needs to be investigated. In this paper, we train a TF-GridNet speech enhancement model and apply it to low-resource datasets that were collected for the ASR task, then train a discrete unit based TTS model on the enhanced speech. We use Arabic datasets as an example and show that the proposed pipeline significantly improves the low-resource TTS system compared with other baseline methods in terms of ASR WER metric. We also run empirical analysis on the correlation between speech enhancement and TTS performances.