APCodec+: A Spectrum-Coding-Based High-Fidelity and High-Compression-Rate Neural Audio Codec with Staged Training Paradigm

TL;DR

APCodec+ is a neural audio codec that uses spectral coding and a staged training approach to achieve high-fidelity, high-compression audio at low bitrates, outperforming traditional codecs.

Contribution

It introduces a novel two-stage training paradigm for spectral-based neural audio codecs, enhancing decoding fidelity and compression efficiency.

Findings

Achieves comparable quality to higher-bitrate codecs at low bitrates.

Improves audio fidelity through staged training of encoder, decoder, and discriminator.

Demonstrates effectiveness of spectral coding with adversarial training.

Abstract

This paper proposes a novel neural audio codec, named APCodec+, which is an improved version of APCodec. The APCodec+ takes the audio amplitude and phase spectra as the coding object, and employs an adversarial training strategy. Innovatively, we propose a two-stage joint-individual training paradigm for APCodec+. In the joint training stage, the encoder, quantizer, decoder and discriminator are jointly trained with complete spectral loss, quantization loss, and adversarial loss. In the individual training stage, the encoder and quantizer fix their parameters and provide high-quality training data for the decoder and discriminator. The decoder and discriminator are individually trained from scratch without the quantization loss. The purpose of introducing individual training is to reduce the learning difficulty of the decoder, thereby further improving the fidelity of the decoded audio. Experimental results confirm that our proposed APCodec+ at low bitrates achieves comparable performance with baseline codecs at higher bitrates, thanks to the proposed staged training paradigm.