Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction

TL;DR

This paper provides a comprehensive, self-contained introduction to diffusion models across both continuous and discrete state spaces, unifying their theoretical foundations and training methods.

Contribution

It develops a unified framework for diffusion processes in general state spaces, connecting continuous SDEs and discrete Markov chains with explicit derivations and training principles.

Findings

Unified diffusion framework for continuous and discrete spaces

Explicit derivation of Fokker-Planck and master equations

Insights into how forward corruption shapes reverse dynamics

Abstract

Although diffusion models now occupy a central place in generative modeling, introductory treatments commonly assume Euclidean data and seldom clarify their connection to discrete-state analogues. This article is a self-contained primer on diffusion over general state spaces, unifying continuous domains and discrete/categorical structures under one lens. We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits -- stochastic differential equations (SDEs) in $\mathbb{R}^d$ and continuous-time Markov chains (CTMCs) on finite alphabets -- and derive the associated Fokker--Planck and master equations. A common variational treatment yields the ELBO that underpins standard training losses. We make explicit how forward corruption choices -- Gaussian processes in continuous spaces and structured categorical transition kernels (uniform, masking/absorbing and more) in discrete spaces -- shape reverse dynamics and the ELBO. The presentation is layered for three audiences: newcomers seeking a self-contained intuitive introduction; diffusion practitioners wanting a global theoretical synthesis; and continuous-diffusion experts looking for an analogy-first path into discrete diffusion. The result is a unified roadmap to modern diffusion methodology across continuous domains and discrete sequences, highlighting a compact set of reusable proofs, identities, and core theoretical principles.