Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models

TL;DR

Warm Diffusion introduces a unified model that combines blurring and noise in diffusion processes, leveraging spectral analysis to improve image generation by balancing randomness and correlation.

Contribution

This paper proposes the Warm Diffusion model, integrating hot and cold diffusion paradigms through spectral analysis for improved image generation.

Findings

Effective image generation across benchmarks

Spectral analysis clarifies the trade-off between noise and blur

Disentangling denoising and deblurring simplifies training

Abstract

Diffusion probabilistic models have achieved remarkable success in generative tasks across diverse data types. While recent studies have explored alternative degradation processes beyond Gaussian noise, this paper bridges two key diffusion paradigms: hot diffusion, which relies entirely on noise, and cold diffusion, which uses only blurring without noise. We argue that hot diffusion fails to exploit the strong correlation between high-frequency image detail and low-frequency structures, leading to random behaviors in the early steps of generation. Conversely, while cold diffusion leverages image correlations for prediction, it neglects the role of noise (randomness) in shaping the data manifold, resulting in out-of-manifold issues and partially explaining its performance drop. To integrate both strengths, we propose Warm Diffusion, a unified Blur-Noise Mixture Diffusion Model (BNMD), to control blurring and noise jointly. Our divide-and-conquer strategy exploits the spectral dependency in images, simplifying score model estimation by disentangling the denoising and deblurring processes. We further analyze the Blur-to-Noise Ratio (BNR) using spectral analysis to investigate the trade-off between model learning dynamics and changes in the data manifold. Extensive experiments across benchmarks validate the effectiveness of our approach for image generation.