Parallel Synthesis for Autoregressive Speech Generation

TL;DR

This paper introduces a novel parallel autoregressive speech synthesis method that significantly improves inference speed by generating speech in frequency subbands and bits, outperforming traditional autoregressive models in quality and efficiency.

Contribution

The paper proposes frequency-wise and bit-wise autoregressive generation techniques, reducing inference time from being proportional to utterance length to being dependent on subbands and bits, enhancing efficiency.

Findings

Achieves faster-than-real-time speech synthesis without GPU acceleration.

Outperforms baseline vocoders in MUSHRA quality scores.

Shows strong generalization to unseen speakers and high sampling rates.

Abstract

Autoregressive neural vocoders have achieved outstanding performance in speech synthesis tasks such as text-to-speech and voice conversion. An autoregressive vocoder predicts a sample at some time step conditioned on those at previous time steps. Though it synthesizes natural human speech, the iterative generation inevitably makes the synthesis time proportional to the utterance length, leading to low efficiency. Many works were dedicated to generating the whole speech sequence in parallel and proposed GAN-based, flow-based, and score-based vocoders. This paper proposed a new thought for the autoregressive generation. Instead of iteratively predicting samples in a time sequence, the proposed model performs frequency-wise autoregressive generation (FAR) and bit-wise autoregressive generation (BAR) to synthesize speech. In FAR, a speech utterance is split into frequency subbands, and a subband is generated conditioned on the previously generated one. Similarly, in BAR, an 8-bit quantized signal is generated iteratively from the first bit. By redesigning the autoregressive method to compute in domains other than the time domain, the number of iterations in the proposed model is no longer proportional to the utterance length but to the number of subbands/bits, significantly increasing inference efficiency. Besides, a post-filter is employed to sample signals from output posteriors; its training objective is designed based on the characteristics of the proposed methods. Experimental results show that the proposed model can synthesize speech faster than real-time without GPU acceleration. Compared with baseline vocoders, the proposed model achieves better MUSHRA results and shows good generalization ability for unseen speakers and 44 kHz speech.