An explicit formulation of the learned noise predictor $\epsilon_{\theta}({\bf x}_t, t)$ via the forward-process noise $\epsilon_{t}$ in denoising diffusion probabilistic models (DDPMs)

TL;DR

This paper derives an explicit formulation of the learned noise predictor in DDPMs in terms of the forward-process noise, providing new theoretical insights and a rigorous proof of a key equality in diffusion models.

Contribution

It introduces a novel explicit formulation of the noise predictor in DDPMs and offers a rigorous proof of a fundamental equality, deepening theoretical understanding.

Findings

Explicit formulation of the noise predictor in terms of forward-process noise

Rigorous proof of the fundamental equality in diffusion models

Enhanced theoretical understanding of diffusion process structure

Abstract

In denoising diffusion probabilistic models (DDPMs), the learned noise predictor $ \epsilon_{\theta} ( {\bf x}_t , t)$ is trained to approximate the forward-process noise $\epsilon_t$. The equality $\nabla_{{\bf x}_t} \log q({\bf x}_t) = -\frac 1 {\sqrt {1- {\bar \alpha}_t} } \epsilon_{\theta} ( {\bf x}_t , t)$ plays a fundamental role in both theoretical analyses and algorithmic design, and thus is frequently employed across diffusion-based generative models. In this paper, an explicit formulation of $ \epsilon_{\theta} ( {\bf x}_t , t)$ in terms of the forward-process noise $\epsilon_t$ is derived. This result show how the forward-process noise $\epsilon_t$ contributes to the learned predictor $ \epsilon_{\theta} ( {\bf x}_t , t)$. Furthermore, based on this formulation, we present a novel and mathematically rigorous proof of the fundamental equality above, clarifying its origin and providing new theoretical insight into the structure of diffusion models.