What Matters in Practical Learned Image Compression

TL;DR

This paper introduces a practical learned image codec optimized for perceptual quality and runtime, achieving significant bitrate savings and real-time performance on mobile devices.

Contribution

It presents a comprehensive study and neural architecture search to design a new codec that balances perceptual quality, speed, and compression efficiency.

Findings

Achieves 2.3-3x bitrate savings over AV1, AV2, VVC, ECM, JPEG-AI.

Provides 20-40% bitrate savings over top learned codecs.

Encodes 12MP images in 230ms and decodes in 150ms on an iPhone 17 Pro Max.

Abstract

One of the major differentiators unlocked by learned codecs relative to their hard-coded traditional counterparts is their ability to be optimized directly to appeal to the human visual system. Despite this potential, a perceptual yet practical image codec is yet to be proposed. In this work, we aim to close this gap. We conduct a comprehensive study of the key modeling choices that govern the design of a practical learned image codec, jointly optimized for perceptual quality and runtime -- including within the ablations several novel techniques. We then perform performance-aware neural architecture search over millions of backbone configurations to identify models that achieve the target on-device runtime while maximizing compression performance as captured by perceptual metrics. We combine the various optimizations to construct a new codec that achieves a significantly improved tradeoff between speed and perceptual quality. Based on rigorous subjective user studies, it provides 2.3-3x bitrate savings against AV1, AV2, VVC, ECM and JPEG-AI, and 20-40% bitrate savings against the best learned codec alternatives. At the same time, on an iPhone 17 Pro Max, it encodes 12MP images as fast as 230ms, and decodes them in 150ms -- faster than most top ML-based codecs run on a V100 GPU.