Puffin: pitch-synchronous neural waveform generation for fullband speech on modest devices

TL;DR

This paper introduces Puffin, a neural vocoder optimized for low-powered devices that synthesizes high-quality 48kHz speech efficiently by combining modern neural techniques with traditional methods.

Contribution

Puffin is a novel neural vocoder that achieves high-quality fullband speech synthesis on modest devices by integrating pitch-synchronous processing and adversarial training.

Findings

Achieves comparable quality to HiFi-GAN baseline

Reduces computational complexity significantly

Produces 48kHz speech on low-powered devices

Abstract

We present a neural vocoder designed with low-powered Alternative and Augmentative Communication devices in mind. By combining elements of successful modern vocoders with established ideas from an older generation of technology, our system is able to produce high quality synthetic speech at 48kHz on devices where neural vocoders are otherwise prohibitively complex. The system is trained adversarially using differentiable pitch synchronous overlap add, and reduces complexity by relying on pitch synchronous Inverse Short-Time Fourier Transform (ISTFT) to generate speech samples. Our system achieves comparable quality with a strong (HiFi-GAN) baseline while using only a fraction of the compute. We present results of a perceptual evaluation as well as an analysis of system complexity.