A Universal Harmonic Discriminator for High-quality GAN-based Vocoder

TL;DR

This paper introduces a universal harmonic discriminator with learnable filters for GAN-based vocoders, enhancing harmonic modeling and dynamic frequency resolution, leading to improved speech and singing synthesis quality.

Contribution

It proposes a novel harmonic filter with learnable triangular band-pass filters and a half-harmonic component, improving harmonic tracking in GAN vocoders.

Findings

Enhanced subjective and objective quality in speech synthesis

Better harmonic modeling for singing voices

Improved performance over traditional STFT-based discriminators

Abstract

With the emergence of GAN-based vocoders, the discriminator, as a crucial component, has been developed recently. In our work, we focus on improving the time-frequency based discriminator. Particularly, Short-Time Fourier Transform (STFT) representation is usually used as input of time-frequency based discriminator. However, the STFT spectrogram has the same frequency resolution at different frequency bins, which results in an inferior performance, especially for singing voices. Motivated by this, we propose a universal harmonic discriminator for dynamic frequency resolution modeling and harmonic tracking. Specifically, we design a harmonic filter with learnable triangular band-pass filter banks, where each frequency bin has a flexible bandwidth. Additionally, we add a half-harmonic to capture fine-grained harmonic relationships at low-frequency band. Experiments on speech and singing datasets validate the effectiveness of the proposed discriminator on both subjective and objective metrics.