Extreme Image Compression using Fine-tuned VQGANs

TL;DR

This paper introduces a vector quantization-based generative image compression framework that achieves extremely low bitrates with high perceptual quality, outperforming existing codecs especially at bitrates below 0.04 bpp.

Contribution

The work integrates VQGANs with clustering and transformers to enable efficient, variable bitrate image compression at extreme low bitrates, with robust image restoration capabilities.

Findings

Outperforms state-of-the-art codecs at bitrates ≤ 0.04 bpp.

Maintains high perceptual quality even with 20% index loss.

Effective image restoration using a transformer in unstable environments.

Abstract

Recent advances in generative compression methods have demonstrated remarkable progress in enhancing the perceptual quality of compressed data, especially in scenarios with low bitrates. However, their efficacy and applicability to achieve extreme compression ratios ($<0.05$ bpp) remain constrained. In this work, we propose a simple yet effective coding framework by introducing vector quantization (VQ)--based generative models into the image compression domain. The main insight is that the codebook learned by the VQGAN model yields a strong expressive capacity, facilitating efficient compression of continuous information in the latent space while maintaining reconstruction quality. Specifically, an image can be represented as VQ-indices by finding the nearest codeword, which can be encoded using lossless compression methods into bitstreams. We propose clustering a pre-trained large-scale codebook into smaller codebooks through the K-means algorithm, yielding variable bitrates and different levels of reconstruction quality within the coding framework. Furthermore, we introduce a transformer to predict lost indices and restore images in unstable environments. Extensive qualitative and quantitative experiments on various benchmark datasets demonstrate that the proposed framework outperforms state-of-the-art codecs in terms of perceptual quality-oriented metrics and human perception at extremely low bitrates ($\le 0.04$ bpp). Remarkably, even with the loss of up to $20\%$ of indices, the images can be effectively restored with minimal perceptual loss.