Quasi-Periodic Parallel WaveGAN: A Non-autoregressive Raw Waveform Generative Model with Pitch-dependent Dilated Convolution Neural Network

TL;DR

This paper introduces QPPWG, a non-autoregressive waveform generator that incorporates pitch-dependent structures to enhance pitch control and speech quality, outperforming previous models in both objective and subjective evaluations.

Contribution

The paper presents a novel quasi-periodic structure with pitch-dependent dilated convolutions in WaveGAN, improving pitch controllability and interpretability in raw waveform speech synthesis.

Findings

QPPWG outperforms PWG in scaled pitch scenarios.

QPPWG demonstrates better spectral and excitation modeling.

Enhanced interpretability of intermediate network outputs.

Abstract

In this paper, we propose a quasi-periodic parallel WaveGAN (QPPWG) waveform generative model, which applies a quasi-periodic (QP) structure to a parallel WaveGAN (PWG) model using pitch-dependent dilated convolution networks (PDCNNs). PWG is a small-footprint GAN-based raw waveform generative model, whose generation time is much faster than real time because of its compact model and non-autoregressive (non-AR) and non-causal mechanisms. Although PWG achieves high-fidelity speech generation, the generic and simple network architecture lacks pitch controllability for an unseen auxiliary fundamental frequency ($F_{0}$) feature such as a scaled $F_{0}$. To improve the pitch controllability and speech modeling capability, we apply a QP structure with PDCNNs to PWG, which introduces pitch information to the network by dynamically changing the network architecture corresponding to the auxiliary $F_{0}$ feature. Both objective and subjective experimental results show that QPPWG outperforms PWG when the auxiliary $F_{0}$ feature is scaled. Moreover, analyses of the intermediate outputs of QPPWG also show better tractability and interpretability of QPPWG, which respectively models spectral and excitation-like signals using the cascaded fixed and adaptive blocks of the QP structure.