CoD: A Diffusion Foundation Model for Image Compression

TL;DR

CoD is a new diffusion foundation model specifically designed for image compression, enabling high efficiency and low-cost training, outperforming previous methods especially at ultra-low bitrates.

Contribution

Introduces CoD, the first compression-oriented diffusion foundation model trained from scratch for end-to-end optimization, surpassing existing codecs at ultra-low bitrates.

Findings

Achieves state-of-the-art results at 0.0039 bpp

Training is 300 times faster than Stable Diffusion

Pixel-space diffusion can outperform GAN-based codecs with fewer parameters

Abstract

Existing diffusion codecs typically build on text-to-image diffusion foundation models like Stable Diffusion. However, text conditioning is suboptimal from a compression perspective, hindering the potential of downstream diffusion codecs, particularly at ultra-low bitrates. To address it, we introduce \textbf{CoD}, the first \textbf{Co}mpression-oriented \textbf{D}iffusion foundation model, trained from scratch to enable end-to-end optimization of both compression and generation. CoD is not a fixed codec but a general foundation model designed for various diffusion-based codecs. It offers several advantages: \textbf{High compression efficiency}, replacing Stable Diffusion with CoD in downstream codecs like DiffC achieves SOTA results, especially at ultra-low bitrates (e.g., 0.0039 bpp); \textbf{Low-cost and reproducible training}, 300$\times$ faster training than Stable Diffusion ($\sim$ 20 vs. $\sim$ 6,250 A100 GPU days) on entirely open image-only datasets; \textbf{Providing new insights}, e.g., We find pixel-space diffusion can achieve VTM-level PSNR with high perceptual quality and can outperform GAN-based codecs using fewer parameters. We hope CoD lays the foundation for future diffusion codec research. Codes are released at https://github.com/microsoft/GenCodec/tree/main/CoD.