SDXL-Lightning: Progressive Adversarial Diffusion Distillation

TL;DR

SDXL-Lightning introduces a novel diffusion distillation technique that combines progressive and adversarial methods to significantly improve one-step and few-step high-resolution text-to-image generation, setting new state-of-the-art results.

Contribution

The paper presents a new diffusion distillation approach that balances quality and diversity, along with theoretical insights, discriminator design, and training strategies, and releases open-source models.

Findings

Achieves state-of-the-art one-step/few-step 1024px text-to-image generation.

Demonstrates effective balance between image quality and mode coverage.

Provides open-source models for broader adoption.

Abstract

We propose a diffusion distillation method that achieves new state-of-the-art in one-step/few-step 1024px text-to-image generation based on SDXL. Our method combines progressive and adversarial distillation to achieve a balance between quality and mode coverage. In this paper, we discuss the theoretical analysis, discriminator design, model formulation, and training techniques. We open-source our distilled SDXL-Lightning models both as LoRA and full UNet weights.