Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion

TL;DR

This paper demonstrates that pixel-space diffusion models can achieve state-of-the-art image synthesis quality at high resolutions, rivaling latent models in efficiency and performance through a simple scaling recipe.

Contribution

The authors introduce SiD2, a pixel-space diffusion model that attains competitive high-resolution image synthesis results with a straightforward scaling approach and architectural simplifications.

Findings

Achieved 1.5 FID on ImageNet512.

Set new SOTA on ImageNet128, ImageNet256, Kinetics600.

Pixel-space models can be as efficient and high-quality as latent models.

Abstract

Latent diffusion models have become the popular choice for scaling up diffusion models for high resolution image synthesis. Compared to pixel-space models that are trained end-to-end, latent models are perceived to be more efficient and to produce higher image quality at high resolution. Here we challenge these notions, and show that pixel-space models can be very competitive to latent models both in quality and efficiency, achieving 1.5 FID on ImageNet512 and new SOTA results on ImageNet128, ImageNet256 and Kinetics600. We present a simple recipe for scaling end-to-end pixel-space diffusion models to high resolutions. 1: Use the sigmoid loss-weighting (Kingma & Gao, 2023) with our prescribed hyper-parameters. 2: Use our simplified memory-efficient architecture with fewer skip-connections. 3: Scale the model to favor processing the image at a high resolution with fewer parameters, rather than using more parameters at a lower resolution. Combining these with guidance intervals, we obtain a family of pixel-space diffusion models we call Simpler Diffusion (SiD2).