Generative Diffusion Modeling: A Practical Handbook

TL;DR

This handbook provides a comprehensive, practical overview of diffusion models, standardizing notation and implementation to aid researchers and practitioners in developing and comparing generative diffusion-based models.

Contribution

It offers a unified perspective, standardized notation, and practical guidance on diffusion models, bridging the gap between research papers and code implementations.

Findings

Standardized notation improves clarity and reproducibility.

Guidelines facilitate robust implementation and fair comparison.

Coverage includes fundamentals, pre-training, and post-training techniques.

Abstract

This handbook offers a unified perspective on diffusion models, encompassing diffusion probabilistic models, score-based generative models, consistency models, rectified flow, and related methods. By standardizing notations and aligning them with code implementations, it aims to bridge the "paper-to-code" gap and facilitate robust implementations and fair comparisons. The content encompasses the fundamentals of diffusion models, the pre-training process, and various post-training methods. Post-training techniques include model distillation and reward-based fine-tuning. Designed as a practical guide, it emphasizes clarity and usability over theoretical depth, focusing on widely adopted approaches in generative modeling with diffusion models.