The Gap Between Principle and Practice of Lossy Image Coding

TL;DR

This paper investigates the discrepancy between the theoretical rate-distortion limit and practical learned image coding schemes, identifying key effects causing the gap and evaluating their impact through simulations.

Contribution

It systematically analyzes the causes of the gap between ideal and empirical rate-distortion functions in learned image coding, highlighting areas for future improvement.

Findings

The gap is caused by five effects: modeling, approximation, amortization, digitization, and asymptotic.

Simulations show the significant impact of the last three effects on performance.

Future coding technologies have high potential to close the gap.

Abstract

Lossy image coding is the art of computing that is principally bounded by the image's rate-distortion function. This bound, though never accurately characterized, has been approached practically via deep learning technologies in recent years. Indeed, learned image coding schemes allow direct optimization of the joint rate-distortion cost, thereby outperforming the handcrafted image coding schemes by a large margin. Still, it is observed that there is room for further improvement in the rate-distortion performance of learned image coding. In this article, we identify the gap between the ideal rate-distortion function forecasted by Shannon's information theory and the empirical rate-distortion function achieved by the state-of-the-art learned image coding schemes, revealing that the gap is incurred by five different effects: modeling effect, approximation effect, amortization effect, digitization effect, and asymptotic effect. We design simulations and experiments to quantitively evaluate the last three effects, which demonstrates the high potential of future lossy image coding technologies.