Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generation

TL;DR

This paper introduces a data-free distillation method for diffusion-based text-to-image models, significantly improving efficiency and performance without access to original training data, achieving state-of-the-art results.

Contribution

The authors propose a novel data-free guided distillation approach with Long and Short Classifier-Free Guidance for efficient one-step text-to-image generation.

Findings

Achieves a record low FID of 8.15 on COCO-2014 without real data.

Rapidly improves image quality and alignment with text using synthetic data.

Maintains competitive CLIP scores while significantly reducing generation time.

Abstract

Diffusion-based text-to-image generation models trained on extensive text-image pairs have demonstrated the ability to produce photorealistic images aligned with textual descriptions. However, a significant limitation of these models is their slow sample generation process, which requires iterative refinement through the same network. To overcome this, we introduce a data-free guided distillation method that enables the efficient distillation of pretrained Stable Diffusion models without access to the real training data, often restricted due to legal, privacy, or cost concerns. This method enhances Score identity Distillation (SiD) with Long and Short Classifier-Free Guidance (LSG), an innovative strategy that applies Classifier-Free Guidance (CFG) not only to the evaluation of the pretrained diffusion model but also to the training and evaluation of the fake score network. We optimize a model-based explicit score matching loss using a score-identity-based approximation alongside our proposed guidance strategies for practical computation. By exclusively training with synthetic images generated by its one-step generator, our data-free distillation method rapidly improves FID and CLIP scores, achieving state-of-the-art FID performance while maintaining a competitive CLIP score. Notably, the one-step distillation of Stable Diffusion 1.5 achieves an FID of 8.15 on the COCO-2014 validation set, a record low value under the data-free setting. Our code and checkpoints are available at https://github.com/mingyuanzhou/SiD-LSG.