A Malliavin calculus approach to score functions in diffusion generative models

TL;DR

This paper introduces a novel, exact formula for the score function in diffusion generative models using Malliavin calculus, improving theoretical understanding and practical estimation methods for complex data distributions.

Contribution

It derives a closed-form expression for the score function in nonlinear diffusion models by combining Malliavin calculus with a Bismut-type formula, eliminating derivatives for better applicability.

Findings

Provides a systematic way to compute score functions using Malliavin derivatives.

Enhances the theoretical foundation for score estimation in diffusion models.

Facilitates the development of new sampling algorithms for complex distributions.

Abstract

Score-based diffusion generative models have recently emerged as a powerful tool for modelling complex data distributions. These models aim at learning the score function, which defines a map from a known probability distribution to the target data distribution via deterministic or stochastic differential equations (SDEs). The score function is typically estimated from data using a variety of approximation techniques, such as denoising or sliced score matching, Hyv\"arien's method, or Schr\"odinger bridges. In this paper, we derive an exact, closed-form, expression for the score function for a broad class of nonlinear diffusion generative models. Our approach combines modern stochastic analysis tools such as Malliavin derivatives and their adjoint operators (Skorokhod integrals or Malliavin Divergence) with a new Bismut-type formula. The resulting expression for the score function can be written entirely in terms of the first and second variation processes, with all Malliavin derivatives systematically eliminated, thereby enhancing its practical applicability. The theoretical framework presented in this work offers a principled foundation for advancing score estimation methods in generative modelling, enabling the design of new sampling algorithms for complex probability distributions. Our results can be extended to broader classes of stochastic differential equations, opening new directions for the development of score-based diffusion generative models.