FlowMAC: Conditional Flow Matching for Audio Coding at Low Bit Rates

TL;DR

FlowMAC is a new neural audio codec using conditional flow matching that achieves high-quality audio compression at low bit rates, offering a scalable, efficient, and tunable solution suitable for real-time applications.

Contribution

This work introduces the first application of conditional flow matching to general audio coding, combining high quality, scalability, and efficiency in a single neural codec.

Findings

FlowMAC at 3 kbps matches higher bit rate GAN and DDPM codecs in quality.

FlowMAC enables real-time CPU audio coding with adjustable complexity.

The model demonstrates scalable and memory-efficient training for low bit rate audio compression.

Abstract

This paper introduces FlowMAC, a novel neural audio codec for high-quality general audio compression at low bit rates based on conditional flow matching (CFM). FlowMAC jointly learns a mel spectrogram encoder, quantizer and decoder. At inference time the decoder integrates a continuous normalizing flow via an ODE solver to generate a high-quality mel spectrogram. This is the first time that a CFM-based approach is applied to general audio coding, enabling a scalable, simple and memory efficient training. Our subjective evaluations show that FlowMAC at 3 kbps achieves similar quality as state-of-the-art GAN-based and DDPM-based neural audio codecs at double the bit rate. Moreover, FlowMAC offers a tunable inference pipeline, which permits to trade off complexity and quality. This enables real-time coding on CPU, while maintaining high perceptual quality.