PitchFlower: A flow-based neural audio codec with pitch controllability

TL;DR

PitchFlower is a novel flow-based neural audio codec that enables explicit pitch control and high-quality speech synthesis by disentangling pitch from other speech attributes during training.

Contribution

It introduces a simple perturbation method for disentangling pitch, combining flow-based decoding with vector quantization to improve controllability and audio quality.

Findings

Achieves more accurate pitch control than WORLD.

Outperforms SiFiGAN in controllability with comparable quality.

Provides a framework for disentangling speech attributes.

Abstract

We present PitchFlower, a flow-based neural audio codec with explicit pitch controllability. Our approach enforces disentanglement through a simple perturbation: during training, F0 contours are flattened and randomly shifted, while the true F0 is provided as conditioning. A vector-quantization bottleneck prevents pitch recovery, and a flow-based decoder generates high quality audio. Experiments show that PitchFlower achieves more accurate pitch control than WORLD at much higher audio quality, and outperforms SiFiGAN in controllability while maintaining comparable quality. Beyond pitch, this framework provides a simple and extensible path toward disentangling other speech attributes.