Is GAN Necessary for Mel-Spectrogram-based Neural Vocoder?

TL;DR

This paper introduces FreeGAN, a neural vocoder that generates high-quality speech without using GANs, improving training efficiency and reducing complexity while maintaining comparable speech quality.

Contribution

The paper proposes a novel GAN-free neural vocoder architecture with amplitude-phase serial prediction, demonstrating comparable performance to GAN-based models.

Findings

FreeGAN achieves speech quality comparable to GAN-based vocoders.

Training efficiency and model complexity are significantly improved.

GAN is not necessary for high-quality mel-spectrogram-based neural vocoding.

Abstract

Recently, mainstream mel-spectrogram-based neural vocoders rely on generative adversarial network (GAN) for high-fidelity speech generation, e.g., HiFi-GAN and BigVGAN. However, the use of GAN restricts training efficiency and model complexity. Therefore, this paper proposes a novel FreeGAN vocoder, aiming to answer the question of whether GAN is necessary for mel-spectrogram-based neural vocoders. The FreeGAN employs an amplitude-phase serial prediction framework, eliminating the need for GAN training. It incorporates amplitude prior input, SNAKE-ConvNeXt v2 backbone and frequency-weighted anti-wrapping phase loss to compensate for the performance loss caused by the absence of GAN. Experimental results confirm that the speech quality of FreeGAN is comparable to that of advanced GAN-based vocoders, while significantly improving training efficiency and complexity. Other explicit-phase-prediction-based neural vocoders can also work without GAN, leveraging our proposed methods.