On Uniform Scalar Quantization for Learned Image Compression

TL;DR

This paper analyzes uniform scalar quantization in learned image compression, revealing a tradeoff between train-test mismatch and gradient estimation risk, and proposes a stochastic annealing method with tricks to improve training stability and performance.

Contribution

It provides a theoretical comparison of quantization surrogates and introduces a stochastic uniform annealing method with practical tricks for better training.

Findings

The tradeoff between train-test mismatch and gradient risk varies with network structure.

The proposed method outperforms existing quantization surrogate practices.

Setting a lower bound on variance reduces train-test mismatch.

Abstract

Learned image compression possesses a unique challenge when incorporating non-differentiable quantization into the gradient-based training of the networks. Several quantization surrogates have been proposed to fulfill the training, but they were not systematically justified from a theoretical perspective. We fill this gap by contrasting uniform scalar quantization, the most widely used category with rounding being its simplest case, and its training surrogates. In principle, we find two factors crucial: one is the discrepancy between the surrogate and rounding, leading to train-test mismatch; the other is gradient estimation risk due to the surrogate, which consists of bias and variance of the gradient estimation. Our analyses and simulations imply that there is a tradeoff between the train-test mismatch and the gradient estimation risk, and the tradeoff varies across different network structures. Motivated by these analyses, we present a method based on stochastic uniform annealing, which has an adjustable temperature coefficient to control the tradeoff. Moreover, our analyses enlighten us as to two subtle tricks: one is to set an appropriate lower bound for the variance parameter of the estimated quantized latent distribution, which effectively reduces the train-test mismatch; the other is to use zero-center quantization with partial stop-gradient, which reduces the gradient estimation variance and thus stabilize the training. Our method with the tricks is verified to outperform the existing practices of quantization surrogates on a variety of representative image compression networks.