Universality of Gaussian-Mixture Reverse Kernels in Conditional Diffusion

TL;DR

This paper proves that certain Gaussian-mixture-based conditional diffusion models can approximate target distributions arbitrarily well, with errors diminishing as the diffusion horizon increases, under specific assumptions.

Contribution

It introduces a theoretical framework showing the universality of Gaussian-mixture reverse kernels in conditional diffusion models, combining Gaussian-mixture theory with ReLU bounds.

Findings

Models can approximate target distributions arbitrarily well in conditional KL divergence.

Error decomposition reduces the problem to a static density approximation per step.

Neural reverse-kernel class is dense in conditional KL under exact terminal matching.

Abstract

We prove that conditional diffusion models whose reverse kernels are finite Gaussian mixtures with ReLU-network logits can approximate suitably regular target distributions arbitrarily well in context-averaged conditional KL divergence, up to an irreducible terminal mismatch that typically vanishes with increasing diffusion horizon. A path-space decomposition reduces the output error to this mismatch plus per-step reverse-kernel errors; assuming each reverse kernel factors through a finite-dimensional feature map, each step becomes a static conditional density approximation problem, solved by composing Norets' Gaussian-mixture theory with quantitative ReLU bounds. Under exact terminal matching the resulting neural reverse-kernel class is dense in conditional KL.