Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation

TL;DR

This paper introduces Control-GIC, a controllable generative image compression framework that enables fine-grained, dynamic bitrate adaptation while maintaining high reconstruction quality, outperforming recent methods.

Contribution

It presents the first controllable image compression method with dynamic granularity adjustment based on local information density and hierarchical feature reconstruction.

Findings

Achieves highly flexible bitrate control across diverse scenarios.

Outperforms state-of-the-art methods in compression quality.

Demonstrates effective hierarchical feature reconstruction.

Abstract

Although recent generative image compression methods have demonstrated impressive potential in optimizing the rate-distortion-perception trade-off, they still face the critical challenge of flexible rate adaption to diverse compression necessities and scenarios. To overcome this challenge, this paper proposes a Controllable Generative Image Compression framework, termed Control-GIC, the first capable of fine-grained bitrate adaption across a broad spectrum while ensuring high-fidelity and generality compression. Control-GIC is grounded in a VQGAN framework that encodes an image as a sequence of variable-length codes (i.e. VQ-indices), which can be losslessly compressed and exhibits a direct positive correlation with the bitrates. Drawing inspiration from the classical coding principle, we correlate the information density of local image patches with their granular representations. Hence, we can flexibly determine a proper allocation of granularity for the patches to achieve dynamic adjustment for VQ-indices, resulting in desirable compression rates. We further develop a probabilistic conditional decoder capable of retrieving historic encoded multi-granularity representations according to transmitted codes, and then reconstruct hierarchical granular features in the formalization of conditional probability, enabling more informative aggregation to improve reconstruction realism. Our experiments show that Control-GIC allows highly flexible and controllable bitrate adaption where the results demonstrate its superior performance over recent state-of-the-art methods. Code is available at https://github.com/lianqi1008/Control-GIC.