Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

TL;DR

This paper introduces LADD, a novel distillation method that leverages latent diffusion features to enable fast, high-resolution image synthesis with minimal sampling steps, outperforming previous pixel-based approaches.

Contribution

LADD simplifies training and improves performance of diffusion model distillation by using pretrained latent features, enabling efficient high-resolution image synthesis.

Findings

LADD achieves high-quality image synthesis with only four sampling steps.

SD3-Turbo matches state-of-the-art performance in text-to-image generation.

LADD is effective in image editing and inpainting applications.

Abstract

Diffusion models are the main driver of progress in image and video synthesis, but suffer from slow inference speed. Distillation methods, like the recently introduced adversarial diffusion distillation (ADD) aim to shift the model from many-shot to single-step inference, albeit at the cost of expensive and difficult optimization due to its reliance on a fixed pretrained DINOv2 discriminator. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models. This approach simplifies training and enhances performance, enabling high-resolution multi-aspect ratio image synthesis. We apply LADD to Stable Diffusion 3 (8B) to obtain SD3-Turbo, a fast model that matches the performance of state-of-the-art text-to-image generators using only four unguided sampling steps. Moreover, we systematically investigate its scaling behavior and demonstrate LADD's effectiveness in various applications such as image editing and inpainting.