Diffusion models under low-noise regime

TL;DR

This paper investigates diffusion models under low-noise conditions, revealing divergence near data manifolds and exploring factors affecting denoising and score accuracy, with implications for robustness and interpretability.

Contribution

It systematically analyzes diffusion models in low-noise regimes, highlighting divergence behaviors and factors influencing denoising trajectories and score accuracy.

Findings

Models trained on disjoint data diverge near data manifold at low noise

Training set size and data geometry influence denoising trajectories

Insights into model robustness and data distribution learning

Abstract

Recent work on diffusion models proposed that they operate in two regimes: memorization, in which models reproduce their training data, and generalization, in which they generate novel samples. While this has been tested in high-noise settings, the behavior of diffusion models as effective denoisers when the corruption level is small remains unclear. To address this gap, we systematically investigated the behavior of diffusion models under low-noise diffusion dynamics, with implications for model robustness and interpretability. Using (i) CelebA subsets of varying sample sizes and (ii) analytic Gaussian mixture benchmarks, we reveal that models trained on disjoint data diverge near the data manifold even when their high-noise outputs converge. We quantify how training set size, data geometry, and model objective choice shape denoising trajectories and affect score accuracy, providing insights into how these models actually learn representations of data distributions. This work starts to address gaps in our understanding of generative model reliability in practical applications where small perturbations are common.