Spectrally-Guided Diffusion Noise Schedules

TL;DR

This paper introduces a spectral property-based method for designing noise schedules in diffusion models, improving image generation quality especially with fewer steps.

Contribution

It presents a principled approach to create per-instance, spectral-guided noise schedules that reduce redundancy and enhance diffusion model performance.

Findings

Spectrally-guided noise schedules outperform handcrafted ones.

The method improves quality in low-step diffusion sampling.

Theoretical bounds guide optimal noise level selection.

Abstract

Denoising diffusion models are widely used for high-quality image and video generation. Their performance depends on noise schedules, which define the distribution of noise levels applied during training and the sequence of noise levels traversed during sampling. Noise schedules are typically handcrafted and require manual tuning across different resolutions. In this work, we propose a principled way to design per-instance noise schedules for pixel diffusion, based on the image's spectral properties. By deriving theoretical bounds on the efficacy of minimum and maximum noise levels, we design ``tight'' noise schedules that eliminate redundant steps. During inference, we propose to conditionally sample such noise schedules. Experiments show that our noise schedules improve generative quality of single-stage pixel diffusion models, particularly in the low-step regime.