Exploiting Latent Properties to Optimize Neural Codecs

TL;DR

This paper introduces methods to enhance neural codecs by leveraging vector quantization and the entropy gradient, resulting in modest rate savings and improved performance over traditional codecs.

Contribution

The paper proposes using optimal uniform vector quantization and the entropy gradient at the decoder to improve neural codec efficiency and performance.

Findings

Rate savings of 1-3% at the same quality

Entropy gradient improves traditional codecs

Optimal uniform vector quantization outperforms non-uniform scalar quantization

Abstract

End-to-end image and video codecs are becoming increasingly competitive, compared to traditional compression techniques that have been developed through decades of manual engineering efforts. These trainable codecs have many advantages over traditional techniques, such as their straightforward adaptation to perceptual distortion metrics and high performance in specific fields thanks to their learning ability. However, current state-of-the-art neural codecs do not fully exploit the benefits of vector quantization and the existence of the entropy gradient in decoding devices. In this paper, we propose to leverage these two properties (vector quantization and entropy gradient) to improve the performance of off-the-shelf codecs. Firstly, we demonstrate that using non-uniform scalar quantization cannot improve performance over uniform quantization. We thus suggest using predefined optimal uniform vector quantization to improve performance. Secondly, we show that the entropy gradient, available at the decoder, is correlated with the reconstruction error gradient, which is not available at the decoder. We therefore use the former as a proxy to enhance compression performance. Our experimental results show that these approaches save between 1 to 3% of the rate for the same quality across various pretrained methods. In addition, the entropy gradient based solution improves traditional codec performance significantly as well.