Towards Efficient Low-rate Image Compression with Frequency-aware Diffusion Prior Refinement

TL;DR

This paper introduces DiffCR, a novel image compression framework that leverages frequency-aware refinement and a lightweight decoder to achieve high-fidelity reconstruction at low bitrates with significantly improved speed and efficiency.

Contribution

DiffCR proposes a frequency-aware skip estimation and a fast two-step decoding process, enhancing diffusion-based image compression without retraining the diffusion model.

Findings

27.2% BD-rate (LPIPS) bitrate savings

65.1% BD-rate (PSNR) improvement

Over 10x faster than state-of-the-art diffusion compression methods

Abstract

Recent advancements in diffusion-based generative priors have enabled visually plausible image compression at extremely low bit rates. However, existing approaches suffer from slow sampling processes and suboptimal bit allocation due to fragmented training paradigms. In this work, we propose Accelerate \textbf{Diff}usion-based Image Compression via \textbf{C}onsistency Prior \textbf{R}efinement (DiffCR), a novel compression framework for efficient and high-fidelity image reconstruction. At the heart of DiffCR is a Frequency-aware Skip Estimation (FaSE) module that refines the $\epsilon$-prediction prior from a pre-trained latent diffusion model and aligns it with compressed latents at different timesteps via Frequency Decoupling Attention (FDA). Furthermore, a lightweight consistency estimator enables fast \textbf{two-step decoding} by preserving the semantic trajectory of diffusion sampling. Without updating the backbone diffusion model, DiffCR achieves substantial bitrate savings (27.2\% BD-rate (LPIPS) and 65.1\% BD-rate (PSNR)) and over $10\times$ speed-up compared to SOTA diffusion-based compression baselines.