SingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation

TL;DR

SingGAN is a novel GAN-based model that achieves high-fidelity, real-time singing voice synthesis with improved high-frequency detail and stability, surpassing previous neural vocoders in quality and efficiency.

Contribution

SingGAN introduces a new GAN architecture with adaptive feature learning and multi-scale discriminators for high-quality singing voice synthesis, the first of its kind.

Findings

Achieves state-of-the-art MOS of 4.05 in singing voice quality.

Operates at 50x real-time speed on a single GPU.

Generalizes well to unseen singers in mel-spectrogram inversion.

Abstract

Deep generative models have achieved significant progress in speech synthesis to date, while high-fidelity singing voice synthesis is still an open problem for its long continuous pronunciation, rich high-frequency parts, and strong expressiveness. Existing neural vocoders designed for text-to-speech cannot directly be applied to singing voice synthesis because they result in glitches and poor high-frequency reconstruction. In this work, we propose SingGAN, a generative adversarial network designed for high-fidelity singing voice synthesis. Specifically, 1) to alleviate the glitch problem in the generated samples, we propose source excitation with the adaptive feature learning filters to expand the receptive field patterns and stabilize long continuous signal generation; and 2) SingGAN introduces global and local discriminators at different scales to enrich low-frequency details and promote high-frequency reconstruction; and 3) To improve the training efficiency, SingGAN includes auxiliary spectrogram losses and sub-band feature matching penalty loss. To the best of our knowledge, SingGAN is the first work designed toward high-fidelity singing voice vocoding. Our evaluation of SingGAN demonstrates the state-of-the-art results with higher-quality (MOS 4.05) samples. Also, SingGAN enables a sample speed of 50x faster than real-time on a single NVIDIA 2080Ti GPU. We further show that SingGAN generalizes well to the mel-spectrogram inversion of unseen singers, and the end-to-end singing voice synthesis system SingGAN-SVS enjoys a two-stage pipeline to transform the music scores into expressive singing voices. Audio samples are available at \url{https://SingGAN.github.io/}