F0 Modeling In Hmm-Based Speech Synthesis System Using Deep Belief Network

TL;DR

This paper explores the use of Deep Belief Networks to model F0 contours in HMM-based speech synthesis, demonstrating improved performance over traditional clustering methods in Bengali speech.

Contribution

It introduces a novel application of Deep Belief Networks for F0 modeling in speech synthesis, showing their effectiveness compared to existing clustering techniques.

Findings

DBN-DNN models outperform clustering tree methods in F0 contour accuracy.

Deeper DBN architectures with more hidden layers yield better results.

The approach improves both objective and subjective speech quality metrics.

Abstract

In recent years multilayer perceptrons (MLPs) with many hid- den layers Deep Neural Network (DNN) has performed sur- prisingly well in many speech tasks, i.e. speech recognition, speaker verification, speech synthesis etc. Although in the context of F0 modeling these techniques has not been ex- ploited properly. In this paper, Deep Belief Network (DBN), a class of DNN family has been employed and applied to model the F0 contour of synthesized speech which was generated by HMM-based speech synthesis system. The experiment was done on Bengali language. Several DBN-DNN architectures ranging from four to seven hidden layers and up to 200 hid- den units per hidden layer was presented and evaluated. The results were compared against clustering tree techniques pop- ularly found in statistical parametric speech synthesis. We show that from textual inputs DBN-DNN learns a high level structure which in turn improves F0 contour in terms of ob- jective and subjective tests.