Diffusion Models: A Mathematical Introduction

TL;DR

This paper provides a comprehensive, mathematically rigorous introduction to diffusion-based generative models, covering their derivation, variants, and practical algorithms with clear explanations for implementation.

Contribution

It offers a detailed, first-principles derivation of diffusion models, including new insights into likelihood estimation, accelerated sampling, and guidance techniques.

Findings

Derivation of diffusion models from Gaussian properties

Introduction of continuous-time probability-flow ODE

Analysis of classifier and classifier-free guidance methods

Abstract

We present a concise, self-contained derivation of diffusion-based generative models. Starting from basic properties of Gaussian distributions (densities, quadratic expectations, re-parameterisation, products, and KL divergences), we construct denoising diffusion probabilistic models from first principles. This includes the forward noising process, its closed-form marginals, the exact discrete reverse posterior, and the related variational bound. This bound simplifies to the standard noise-prediction goal used in practice. We then discuss likelihood estimation and accelerated sampling, covering DDIM, adversarially learned reverse dynamics (DDGAN), and multi-scale variants such as nested and latent diffusion, with Stable Diffusion as a canonical example. A continuous-time formulation follows, in which we derive the probability-flow ODE from the diffusion SDE via the continuity and Fokker-Planck equations, introduce flow matching, and show how rectified flows recover DDIM up to a time re-parameterisation. Finally, we treat guided diffusion, interpreting classifier guidance as a posterior score correction and classifier-free guidance as a principled interpolation between conditional and unconditional scores. Throughout, the focus is on transparent algebra, explicit intermediate steps, and consistent notation, so that readers can both follow the theory and implement the corresponding algorithms in practice.