Entropy-Based Dimension-Free Convergence and Loss-Adaptive Schedules for Diffusion Models

TL;DR

This paper introduces a dimension-free, entropy-based convergence analysis for diffusion models and proposes a loss-adaptive schedule that enhances sampling efficiency and quality without geometric assumptions.

Contribution

It develops a novel information-theoretic framework for dimension-free convergence analysis and introduces LAS, a loss-based adaptive schedule for improved diffusion sampling.

Findings

KL divergence bound of O(H^2/K) under mild assumptions

LAS improves sampling quality over heuristic schedules

Method avoids geometric restrictions and scales independently of ambient dimension

Abstract

Diffusion generative models synthesize samples by discretizing reverse-time dynamics driven by a learned score (or denoiser). Existing convergence analyses of diffusion models typically scale at least linearly with the ambient dimension, and sharper rates often depend on intrinsic-dimension assumptions or other geometric restrictions on the target distribution. We develop an alternative, information-theoretic approach to dimension-free convergence that avoids any geometric assumptions. Under mild assumptions on the target distribution, we bound KL divergence between the target and generated distributions by $O(H^2/K)$ (up to endpoint factors), where $H$ is the Shannon entropy and $K$ is the number of sampling steps. Moreover, using a reformulation of the KL divergence, we propose a Loss-Adaptive Schedule (LAS) for efficient discretization of reverse SDE which is lightweight and relies only on the training loss, requiring no post-training heavy computation. Empirically, LAS improves sampling quality over common heuristic schedules.