Why DDIM Hallucinates More than DDPM: A Theoretical Analysis of Reverse Dynamics

TL;DR

This paper provides a theoretical analysis comparing DDIM and DDPM diffusion models, revealing how stochasticity in DDPM reduces hallucinations and suggesting ways to improve deterministic samplers.

Contribution

It offers a theoretical explanation for hallucination differences between DDIM and DDPM, and proposes stochastic steps to mitigate hallucinations in deterministic models.

Findings

DDIM can get stuck on mode segments after a critical time

DDPM's stochasticity helps it escape hallucination regions

Adding stochastic steps to DDIM reduces hallucinations

Abstract

We theoretically study the hallucination phenomena in two canonical diffusion samplers: the stochastic Denoising Diffusion Probabilistic Model (DDPM) and the deterministic Denoising Diffusion Implicit Model (DDIM). We analyze the reverse ODE (DDIM) and SDE (DDPM) for a Gaussian mixture target, proving that after a critical time $\tau$, (a) DDIM can become stuck on the segment connecting the two nearest modes and (b) DDPM *stochasticity* helps it become unstuck from this region, thus avoiding hallucination. Our empirical validation verifies that DDPM has a significantly lower hallucination rate than DDIM when this region is entered. Building on our observations, we exhibit how using additional stochastic steps can help DDIM avoid hallucinations and offer new insights on how to design improved samplers.