Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis

TL;DR

This paper introduces a spectral analysis-based Beta Sampling method for diffusion models, improving image generation efficiency by focusing on critical denoising steps, and demonstrating superior performance over uniform sampling.

Contribution

It proposes a novel Beta Sampling technique guided by spectral analysis to optimize diffusion process steps, enhancing efficiency and output quality.

Findings

Beta Sampling outperforms uniform sampling in FID and IS scores.

Spectral analysis reveals significant low-frequency changes early and high-frequency changes later.

Method achieves competitive efficiency compared to state-of-the-art diffusion techniques.

Abstract

Generative diffusion models have emerged as a powerful tool for high-quality image synthesis, yet their iterative nature demands significant computational resources. This paper proposes an efficient time step sampling method based on an image spectral analysis of the diffusion process, aimed at optimizing the denoising process. Instead of the traditional uniform distribution-based time step sampling, we introduce a Beta distribution-like sampling technique that prioritizes critical steps in the early and late stages of the process. Our hypothesis is that certain steps exhibit significant changes in image content, while others contribute minimally. We validated our approach using Fourier transforms to measure frequency response changes at each step, revealing substantial low-frequency changes early on and high-frequency adjustments later. Experiments with ADM and Stable Diffusion demonstrated that our Beta Sampling method consistently outperforms uniform sampling, achieving better FID and IS scores, and offers competitive efficiency relative to state-of-the-art methods like AutoDiffusion. This work provides a practical framework for enhancing diffusion model efficiency by focusing computational resources on the most impactful steps, with potential for further optimization and broader application.