PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation

TL;DR

PeriodWave is a novel waveform generation model that explicitly captures periodic features using multi-period flow matching, achieving high-fidelity results efficiently in tasks like TTS.

Contribution

It introduces a period-aware flow matching estimator and a multi-period estimator, along with a single period-conditional universal estimator for efficient high-quality waveform synthesis.

Findings

Outperforms previous models in Mel-spectrogram reconstruction

Achieves superior results in text-to-speech tasks

Effectively disentangles frequency information for high-fidelity generation

Abstract

Recently, universal waveform generation tasks have been investigated conditioned on various out-of-distribution scenarios. Although GAN-based methods have shown their strength in fast waveform generation, they are vulnerable to train-inference mismatch scenarios such as two-stage text-to-speech. Meanwhile, diffusion-based models have shown their powerful generative performance in other domains; however, they stay out of the limelight due to slow inference speed in waveform generation tasks. Above all, there is no generator architecture that can explicitly disentangle the natural periodic features of high-resolution waveform signals. In this paper, we propose PeriodWave, a novel universal waveform generation model. First, we introduce a period-aware flow matching estimator that can capture the periodic features of the waveform signal when estimating the vector fields. Additionally, we utilize a multi-period estimator that avoids overlaps to capture different periodic features of waveform signals. Although increasing the number of periods can improve the performance significantly, this requires more computational costs. To reduce this issue, we also propose a single period-conditional universal estimator that can feed-forward parallel by period-wise batch inference. Additionally, we utilize discrete wavelet transform to losslessly disentangle the frequency information of waveform signals for high-frequency modeling, and introduce FreeU to reduce the high-frequency noise for waveform generation. The experimental results demonstrated that our model outperforms the previous models both in Mel-spectrogram reconstruction and text-to-speech tasks. All source code will be available at \url{https://github.com/sh-lee-prml/PeriodWave}.