QuickVC: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform for Faster Conversion

TL;DR

This paper introduces QuickVC, a fast and lightweight voice conversion model that leverages inverse short-time Fourier transform to significantly improve inference speed while maintaining high-quality speech synthesis.

Contribution

The study presents a novel VC model using iSTFT for faster conversion and HuBERT-Soft for content extraction, outperforming existing methods in speed without sacrificing quality.

Findings

Achieves over 5000 kHz sampling rate on GPU

Generates over 250 kHz on CPU

Maintains competitive naturalness and similarity

Abstract

With the development of automatic speech recognition (ASR) and text-to-speech (TTS) technology, high-quality voice conversion (VC) can be achieved by extracting source content information and target speaker information to reconstruct waveforms. However, current methods still require improvement in terms of inference speed. In this study, we propose a lightweight VITS-based VC model that uses the HuBERT-Soft model to extract content information features without speaker information. Through subjective and objective experiments on synthesized speech, the proposed model demonstrates competitive results in terms of naturalness and similarity. Importantly, unlike the original VITS model, we use the inverse short-time Fourier transform (iSTFT) to replace the most computationally expensive part. Experimental results show that our model can generate samples at over 5000 kHz on the 3090 GPU and over 250 kHz on the i9-10900K CPU, achieving competitive speed for the same hardware configuration.