Pretrained Diffusion Models Are Inherently Skipped-Step Samplers

TL;DR

This paper reveals that pretrained diffusion models can inherently skip multiple denoising steps during sampling, enabling faster generation without sacrificing quality, by leveraging their intrinsic properties.

Contribution

The authors introduce a skipped-step sampling mechanism that is derived from the same training objective, demonstrating that accelerated sampling is an inherent property of pretrained diffusion models.

Findings

Achieves high-quality generation with fewer steps

Compatible with models like OpenAI ADM and Stable Diffusion

Significantly reduces sampling time without quality loss

Abstract

Diffusion models have been achieving state-of-the-art results across various generation tasks. However, a notable drawback is their sequential generation process, requiring long-sequence step-by-step generation. Existing methods, such as DDIM, attempt to reduce sampling steps by constructing a class of non-Markovian diffusion processes that maintain the same training objective. However, there remains a gap in understanding whether the original diffusion process can achieve the same efficiency without resorting to non-Markovian processes. In this paper, we provide a confirmative answer and introduce skipped-step sampling, a mechanism that bypasses multiple intermediate denoising steps in the iterative generation process, in contrast with the traditional step-by-step refinement of standard diffusion inference. Crucially, we demonstrate that this skipped-step sampling mechanism is derived from the same training objective as the standard diffusion model, indicating that accelerated sampling via skipped-step sampling via a Markovian way is an intrinsic property of pretrained diffusion models. Additionally, we propose an enhanced generation method by integrating our accelerated sampling technique with DDIM. Extensive experiments on popular pretrained diffusion models, including the OpenAI ADM, Stable Diffusion, and Open Sora models, show that our method achieves high-quality generation with significantly reduced sampling steps.