Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial

TL;DR

This tutorial provides a comprehensive technical overview of score-based diffusion models formulated through stochastic differential equations, covering core concepts, methods, and recent advances for practitioners and researchers.

Contribution

It offers a detailed, accessible introduction to the SDE-based formulation of diffusion models, including sampling, score matching, and related techniques, with illustrative proofs.

Findings

Clarifies the SDE formulation of diffusion models

Explains the relationship between sampling and score matching

Provides insights useful for designing new diffusion algorithms

Abstract

This is an expository article on the score-based diffusion models, with a particular focus on the formulation via stochastic differential equations (SDE). After a gentle introduction, we discuss the two pillars in the diffusion modeling -- sampling and score matching, which encompass the SDE/ODE sampling, score matching efficiency, the consistency models, and reinforcement learning. Short proofs are given to illustrate the main idea of the stated results. The article is primarily a technical introduction to the field, and practitioners may also find some analysis useful in designing new models or algorithms.