Discrete generative diffusion models without stochastic differential equations: a tensor network approach

TL;DR

This paper introduces a novel tensor network-based approach to discrete diffusion models that efficiently generate samples from lattice systems without solving stochastic differential equations, enabling better modeling of complex thermodynamic systems.

Contribution

The authors develop a tensor network framework for discrete diffusion models, allowing exact representation and efficient sampling of lattice systems without stochastic differential equations.

Findings

Tensor networks enable exact denoising dynamics representation.

Auto-regressive tensor networks allow efficient, unbiased sample generation.

The method effectively models equilibrium states of complex thermodynamic systems.

Abstract

Diffusion models (DMs) are a class of generative machine learning methods that sample a target distribution by transforming samples of a trivial (often Gaussian) distribution using a learned stochastic differential equation. In standard DMs, this is done by learning a ``score function'' that reverses the effect of adding diffusive noise to the distribution of interest. Here we consider the generalisation of DMs to lattice systems with discrete degrees of freedom, and where noise is added via Markov chain jump dynamics. We show how to use tensor networks (TNs) to efficiently define and sample such ``discrete diffusion models'' (DDMs) without explicitly having to solve a stochastic differential equation. We show the following: (i) by parametrising the data and evolution operators as TNs, the denoising dynamics can be represented exactly; (ii) the auto-regressive nature of TNs allows to generate samples efficiently and without bias; (iii) for sampling Boltzmann-like distributions, TNs allow to construct an efficient learning scheme that integrates well with Monte Carlo. We illustrate this approach to study the equilibrium of two models with non-trivial thermodynamics, the $d=1$ constrained Fredkin chain and the $d=2$ Ising model.