Latent-Shift: Gradient of Entropy Helps Neural Codecs

TL;DR

This paper introduces a novel technique for neural codecs that leverages the gradient of entropy at the decoder side to improve compression efficiency, achieving 1-2% rate savings without additional complexity.

Contribution

It theoretically links entropy gradient to reconstruction error gradient and demonstrates practical rate savings using this insight in neural image/video codecs.

Findings

Achieves 1-2% rate savings at the same quality.

Theoretically shows entropy gradient correlates with reconstruction error gradient.

Method is orthogonal and complementary to existing improvements.

Abstract

End-to-end image/video codecs are getting competitive compared to traditional compression techniques that have been developed through decades of manual engineering efforts. These trainable codecs have many advantages over traditional techniques such as easy adaptation on perceptual distortion metrics and high performance on specific domains thanks to their learning ability. However, state of the art neural codecs does not take advantage of the existence of gradient of entropy in decoding device. In this paper, we theoretically show that gradient of entropy (available at decoder side) is correlated with the gradient of the reconstruction error (which is not available at decoder side). We then demonstrate experimentally that this gradient can be used on various compression methods, leading to a $1-2\%$ rate savings for the same quality. Our method is orthogonal to other improvements and brings independent rate savings.