Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Prior

TL;DR

This paper introduces a two-stage extreme image compression method that uses pre-trained diffusion models and latent guidance to achieve high-quality reconstructions at very low bitrates, outperforming existing approaches.

Contribution

The work presents a novel framework combining VAE-based compression and diffusion models with a space alignment loss for improved low-bitrate image reconstruction.

Findings

Significantly better visual quality at extremely low bitrates compared to state-of-the-art methods.

Effective use of diffusion models for image reconstruction in compression.

Introduction of a space alignment loss to enhance content diffusion space alignment.

Abstract

Image compression at extremely low bitrates (below 0.1 bits per pixel (bpp)) is a significant challenge due to substantial information loss. In this work, we propose a novel two-stage extreme image compression framework that exploits the powerful generative capability of pre-trained diffusion models to achieve realistic image reconstruction at extremely low bitrates. In the first stage, we treat the latent representation of images in the diffusion space as guidance, employing a VAE-based compression approach to compress images and initially decode the compressed information into content variables. The second stage leverages pre-trained stable diffusion to reconstruct images under the guidance of content variables. Specifically, we introduce a small control module to inject content information while keeping the stable diffusion model fixed to maintain its generative capability. Furthermore, we design a space alignment loss to force the content variables to align with the diffusion space and provide the necessary constraints for optimization. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches in terms of visual performance at extremely low bitrates. The source code and trained models are available at https://github.com/huai-chang/DiffEIC.