Efficient Evaluation of Quantization-Effects in Neural Codecs

TL;DR

This paper introduces a fast, resource-efficient framework for evaluating quantization effects in neural codecs, enabling better understanding and stabilization of training without extensive computational costs.

Contribution

It presents a novel evaluation method using simplified models to analyze quantization effects, improving efficiency and revealing behaviors in neural codecs.

Findings

Efficient evaluation framework reduces training time and computational resources.

Identifies distinct behaviors of neural codecs related to quantization.

Proposes a stabilization technique for the straight-through estimator.

Abstract

Neural codecs, comprising an encoder, quantizer, and decoder, enable signal transmission at exceptionally low bitrates. Training these systems requires techniques like the straight-through estimator, soft-to-hard annealing, or statistical quantizer emulation to allow a non-zero gradient across the quantizer. Evaluating the effect of quantization in neural codecs, like the influence of gradient passing techniques on the whole system, is often costly and time-consuming due to training demands and the lack of affordable and reliable metrics. This paper proposes an efficient evaluation framework for neural codecs using simulated data with a defined number of bits and low-complexity neural encoders/decoders to emulate the non-linear behavior in larger networks. Our system is highly efficient in terms of training time and computational and hardware requirements, allowing us to uncover distinct behaviors in neural codecs. We propose a modification to stabilize training with the straight-through estimator based on our findings. We validate our findings against an internal neural audio codec and against the state-of-the-art descript-audio-codec.