CoD-Lite: Real-Time Diffusion-Based Generative Image Compression

TL;DR

This paper introduces CoD-Lite, a lightweight diffusion-based image codec that achieves real-time performance at 1080p with significant bitrate reduction, suitable for practical applications.

Contribution

It designs a one-step lightweight convolution diffusion codec with distillation and adversarial learning, enabling real-time image compression at high resolution.

Findings

Generation-oriented pre-training is less effective at small scales.

Lightweight convolutions suffice for diffusion-based compression.

The codec achieves 60 FPS encoding and 42 FPS decoding at 1080p.

Abstract

Recent advanced diffusion methods typically derive strong generative priors by scaling diffusion transformers. However, scaling fails to generalize when adapted for real-time compression scenarios that demand lightweight models. In this paper, we explore the design of real-time and lightweight diffusion codecs by addressing two pivotal questions. First, does diffusion pre-training benefit lightweight diffusion codecs? Through systematic analysis, we find that generation-oriented pre-training is less effective at small model scales whereas compression-oriented pre-training yields consistently better performance. Second, are transformers essential? We find that while global attention is crucial for standard generation, lightweight convolutions suffice for compression-oriented diffusion when paired with distillation. Guided by these findings, we establish a one-step lightweight convolution diffusion codec that achieves real-time $60$~FPS encoding and $42$~FPS decoding at 1080p. Further enhanced by distillation and adversarial learning, the proposed codec reduces bitrate by 85\% at a comparable FID to MS-ILLM, bridging the gap between generative compression and practical real-time deployment. Codes are released at https://github.com/microsoft/GenCodec/tree/main/CoD_Lite