Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model

TL;DR

This paper introduces a versatile neural image compression model that allows users to control rate, distortion, and realism simultaneously within a single model, outperforming previous methods that required multiple models for different bit rates.

Contribution

The authors develop a novel variable-rate generative NIC model with tailored discriminator designs and a multi-realism technique, enabling ultra-controllable image compression in a single model.

Findings

Achieves comparable or better performance than state-of-the-art single-rate models.

Supports a wide range of bit rates with one model.

Allows user adjustment of rate, distortion, and realism.

Abstract

In recent years, neural network-driven image compression (NIC) has gained significant attention. Some works adopt deep generative models such as GANs and diffusion models to enhance perceptual quality (realism). A critical obstacle of these generative NIC methods is that each model is optimized for a single bit rate. Consequently, multiple models are required to compress images to different bit rates, which is impractical for real-world applications. To tackle this issue, we propose a variable-rate generative NIC model. Specifically, we explore several discriminator designs tailored for the variable-rate approach and introduce a novel adversarial loss. Moreover, by incorporating the newly proposed multi-realism technique, our method allows the users to adjust the bit rate, distortion, and realism with a single model, achieving ultra-controllability. Unlike existing variable-rate generative NIC models, our method matches or surpasses the performance of state-of-the-art single-rate generative NIC models while covering a wide range of bit rates using just one model. Code will be available at https://github.com/iwa-shi/CRDR