On the Importance of Noise Scheduling for Diffusion Models

TL;DR

This paper investigates how different noise scheduling strategies impact the performance of diffusion models, revealing task-dependent optimal schedules and proposing a simple scaling method that improves high-resolution image generation.

Contribution

It provides empirical insights into noise scheduling effects and introduces a simple data scaling strategy that enhances high-resolution diffusion model performance.

Findings

Optimal noise schedules vary with image size and task.

Scaling input data by a factor improves performance across sizes.

Achieved state-of-the-art high-resolution image generation on ImageNet.

Abstract

We empirically study the effect of noise scheduling strategies for denoising diffusion generative models. There are three findings: (1) the noise scheduling is crucial for the performance, and the optimal one depends on the task (e.g., image sizes), (2) when increasing the image size, the optimal noise scheduling shifts towards a noisier one (due to increased redundancy in pixels), and (3) simply scaling the input data by a factor of $b$ while keeping the noise schedule function fixed (equivalent to shifting the logSNR by $\log b$) is a good strategy across image sizes. This simple recipe, when combined with recently proposed Recurrent Interface Network (RIN), yields state-of-the-art pixel-based diffusion models for high-resolution images on ImageNet, enabling single-stage, end-to-end generation of diverse and high-fidelity images at 1024$\times$1024 resolution (without upsampling/cascades).