A Cyclical Approach to Synthetic and Natural Speech Mismatch Refinement of Neural Post-filter for Low-cost Text-to-speech System

TL;DR

This paper introduces a cyclical framework for refining neural post-filters in low-cost TTS systems, effectively addressing acoustic and temporal mismatches using neural vocoders, with demonstrated improvements through evaluations.

Contribution

It proposes a novel cyclical approach to improve neural post-filters for low-cost TTS by tackling acoustic and temporal mismatches with neural vocoders.

Findings

Effective reduction of acoustic mismatch.

Improved speech naturalness in subjective tests.

Framework applicable to low-resource TTS systems.

Abstract

Neural-based text-to-speech (TTS) systems achieve very high-fidelity speech generation because of the rapid neural network developments. However, the huge labeled corpus and high computation cost requirements limit the possibility of developing a high-fidelity TTS system by small companies or individuals. On the other hand, a neural vocoder, which has been widely adopted for the speech generation in neural-based TTS systems, can be trained with a relatively small unlabeled corpus. Therefore, in this paper, we explore a general framework to develop a neural post-filter (NPF) for low-cost TTS systems using neural vocoders. A cyclical approach is proposed to tackle the acoustic and temporal mismatches (AM and TM) of developing an NPF. Both objective and subjective evaluations have been conducted to demonstrate the AM and TM problems and the effectiveness of the proposed framework.