Statistical Convergence of Spherical First Hitting Diffusion Models

TL;DR

This paper proves that First Hitting Diffusion Models (FHDMs) achieve optimal statistical convergence rates for spherical data distributions, marking the first such result for models with random generation time.

Contribution

It establishes the statistical convergence properties of FHDMs, demonstrating their minimax optimality in total variation for certain data distributions.

Findings

FHDMs achieve minimax optimal convergence rates up to logarithmic factors.

First statistical optimality result for denoising diffusion models with random generation time.

FHDMs leverage data manifold information for improved performance.

Abstract

Denoising diffusion models have evolved into a state-of-the-art method for tasks in various fields, such as denoising and generation of images, text generation, or generation of synthetic data for training of other machine learning models. First hitting diffusion models (FHDM) are a particular class of denoising diffusion models with \textit{random} adaptive generation time tailored to generate data on a known manifold. Building on the conditioning framework of Doob's $h$-transform these models leverage the given information on the target data manifold to demonstrate strong performance across tasks while offering distinct features such as time-homogeneous dynamics of the generating process and a reduced average simulation time. Even though the theoretical investigation of standard forward-backward diffusion models has attracted much attention in the recent past, the statistical convergence properties of FHDMs are not yet understood. In this work, we show that, up to logarithmic factors, FHDMs achieve the minimax optimal convergence rate in total variation for spherically supported Sobolev smooth data distributions. In particular, this is the first statistical optimality result for denoising diffusion modelling with random generation time.