A Theoretical Framework for Rate-Distortion Limits in Learned Image Compression

TL;DR

This paper introduces a comprehensive theoretical framework for understanding the rate-distortion limits of learned image compression, decomposing performance loss into key components and providing insights for system design.

Contribution

It offers a novel, interpretable analysis method for learned image codecs, bridging empirical results with information-theoretic limits and guiding future improvements.

Findings

Derived optimal latent variance under Gaussian assumption

Quantified gap between uniform quantization and Gaussian test channel

Demonstrated impact of accurate mean prediction on entropy reduction

Abstract

We present a novel systematic theoretical framework to analyze the rate-distortion (R-D) limits of learned image compression. While recent neural codecs have achieved remarkable empirical results, their distance from the information-theoretic limit remains unclear. Our work addresses this gap by decomposing the R-D performance loss into three key components: variance estimation, quantization strategy, and context modeling. First, we derive the optimal latent variance as the second moment under a Gaussian assumption, providing a principled alternative to hyperprior-based estimation. Second, we quantify the gap between uniform quantization and the Gaussian test channel derived from the reverse water-filling theorem. Third, we extend our framework to include context modeling, and demonstrate that accurate mean prediction yields substantial entropy reduction. Unlike prior R-D estimators, our method provides a structurally interpretable perspective that aligns with real compression modules and enables fine-grained analysis. Through joint simulation and end-to-end training, we derive a tight and actionable approximation of the theoretical R-D limits, offering new insights into the design of more efficient learned compression systems.