Singing voice synthesis based on convolutional neural networks

TL;DR

This paper introduces a CNN-based singing voice synthesis system that models long-term dependencies to produce more natural singing voices, demonstrating improved quality over existing DNN-based methods.

Contribution

The paper proposes a novel CNN architecture for singing voice synthesis that captures long-term dependencies without needing a parameter generation algorithm.

Findings

CNN-based system produces more natural singing voices

Experimental results show improved subjective listening scores

Long-term dependency modeling enhances synthesis quality

Abstract

The present paper describes a singing voice synthesis based on convolutional neural networks (CNNs). Singing voice synthesis systems based on deep neural networks (DNNs) are currently being proposed and are improving the naturalness of synthesized singing voices. In these systems, the relationship between musical score feature sequences and acoustic feature sequences extracted from singing voices is modeled by DNNs. Then, an acoustic feature sequence of an arbitrary musical score is output in units of frames by the trained DNNs, and a natural trajectory of a singing voice is obtained by using a parameter generation algorithm. As singing voices contain rich expression, a powerful technique to model them accurately is required. In the proposed technique, long-term dependencies of singing voices are modeled by CNNs. An acoustic feature sequence is generated in units of segments that consist of long-term frames, and a natural trajectory is obtained without the parameter generation algorithm. Experimental results in a subjective listening test show that the proposed architecture can synthesize natural sounding singing voices.