HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform

TL;DR

HiFTNet is a novel neural vocoder that combines inverse STFT with a harmonic-plus-noise filter, achieving high-quality, fast, and parameter-efficient speech synthesis suitable for real-time applications.

Contribution

It introduces HiFTNet, integrating harmonic-plus-noise filtering with iSTFT for improved speed and quality over prior GAN-based vocoders.

Findings

Outperforms iSTFTNet and HiFi-GAN in subjective quality.

Achieves real-time inference with fewer parameters.

Matches or exceeds BigVGAN performance on benchmark datasets.

Abstract

Recent advancements in speech synthesis have leveraged GAN-based networks like HiFi-GAN and BigVGAN to produce high-fidelity waveforms from mel-spectrograms. However, these networks are computationally expensive and parameter-heavy. iSTFTNet addresses these limitations by integrating inverse short-time Fourier transform (iSTFT) into the network, achieving both speed and parameter efficiency. In this paper, we introduce an extension to iSTFTNet, termed HiFTNet, which incorporates a harmonic-plus-noise source filter in the time-frequency domain that uses a sinusoidal source from the fundamental frequency (F0) inferred via a pre-trained F0 estimation network for fast inference speed. Subjective evaluations on LJSpeech show that our model significantly outperforms both iSTFTNet and HiFi-GAN, achieving ground-truth-level performance. HiFTNet also outperforms BigVGAN-base on LibriTTS for unseen speakers and achieves comparable performance to BigVGAN while being four times faster with only $1/6$ of the parameters. Our work sets a new benchmark for efficient, high-quality neural vocoding, paving the way for real-time applications that demand high quality speech synthesis.