Nonlinear Transform Coding

TL;DR

Nonlinear transform coding (NTC) methods have recently become competitive with traditional linear codecs for images, surpassing them in perceptual quality metrics like MS-SSIM, and this paper reviews their performance, architectures, and optimization techniques.

Contribution

This paper provides a comprehensive review of NTC methods, introduces a novel entropy-constrained vector quantization variant, and compares different parameterizations of the rate-distortion trade-off.

Findings

NTC methods outperform linear codecs in perceptual quality metrics.

A new entropy-constrained vector quantization variant is proposed.

Different NTC architectures and optimization techniques are systematically compared.

Abstract

We review a class of methods that can be collected under the name nonlinear transform coding (NTC), which over the past few years have become competitive with the best linear transform codecs for images, and have superseded them in terms of rate--distortion performance under established perceptual quality metrics such as MS-SSIM. We assess the empirical rate--distortion performance of NTC with the help of simple example sources, for which the optimal performance of a vector quantizer is easier to estimate than with natural data sources. To this end, we introduce a novel variant of entropy-constrained vector quantization. We provide an analysis of various forms of stochastic optimization techniques for NTC models; review architectures of transforms based on artificial neural networks, as well as learned entropy models; and provide a direct comparison of a number of methods to parameterize the rate--distortion trade-off of nonlinear transforms, introducing a simplified one.