U-Turn Diffusion

TL;DR

This paper introduces U-Turn diffusion, a method that modifies pre-trained diffusion models to generate synthetic samples by adjusting the forward and reverse processes.

Contribution

The novel U-Turn diffusion method shortens the forward and reverse processes while maintaining detailed balance and introduces critical times for memorization and speciation.

Findings

Generated samples diverge from the ground truth after a Memorization Time Tm.

At Speciation Time Ts, samples begin representing different classes.

The score function becomes effectively affine for t > Ts.

Abstract

We investigate diffusion models generating synthetic samples from the probability distribution represented by the ground truth (GT) samples. We focus on how GT sample information is encoded in the score function (SF), computed (not simulated) from the Wiener–Ito linear forward process in the artificial time t∈[0→∞], and then used as a nonlinear drift in the simulated Wiener–Ito reverse process with t∈[∞→0]. We propose U-Turn diffusion, an augmentation of a pre-trained diffusion model, which shortens the forward and reverse processes to t∈[0→Tu] and t∈[Tu→0]. The U-Turn reverse process is initialized at Tu with a sample from the probability distribution of the forward process (initialized at t=0 with a GT sample) ensuring a detailed balance relation between the shortened forward and reverse processes. Our experiments on the class-conditioned SF of the ImageNet dataset and the multi-class, single SF of the CIFAR-10 dataset reveal a critical Memorization Time Tm, beyond which generated samples diverge from the GT sample used to initialize the U-Turn scheme, and a Speciation Time Ts, where for Tu>Ts>Tm, samples begin representing different classes. We further examine the role of SF nonlinearity through a Gaussian Test, comparing empirical and Gaussian-approximated U-Turn auto-correlation functions and showing that the SF becomes effectively affine for t>Ts and approximately affine for t∈[Tm,Ts].

Figures