Very Low Complexity Speech Synthesis Using Framewise Autoregressive GAN (FARGAN) with Pitch Prediction

TL;DR

This paper introduces FARGAN, a low-complexity autoregressive neural vocoder that uses pitch prediction to synthesize high-quality speech efficiently, outperforming existing low-complexity methods.

Contribution

FARGAN is a novel autoregressive vocoder leveraging long-term pitch prediction to achieve high-quality speech synthesis with significantly reduced computational complexity.

Findings

FARGAN achieves 600 MFLOPS complexity.

It outperforms existing low-complexity vocoders in quality.

Its quality matches higher-complexity vocoders.

Abstract

Neural vocoders are now being used in a wide range of speech processing applications. In many of those applications, the vocoder can be the most complex component, so finding lower complexity algorithms can lead to significant practical benefits. In this work, we propose FARGAN, an autoregressive vocoder that takes advantage of long-term pitch prediction to synthesize high-quality speech in small subframes, without the need for teacher-forcing. Experimental results show that the proposed 600~MFLOPS FARGAN vocoder can achieve both higher quality and lower complexity than existing low-complexity vocoders. The quality even matches that of existing higher-complexity vocoders.