Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Acceleration

TL;DR

This paper analyzes the Wasserstein convergence of score-based diffusion models, comparing discretization schemes and introducing a Hessian-based acceleration that significantly improves convergence rates.

Contribution

It provides a detailed theoretical analysis of discretization effects and proposes a novel Hessian-informed accelerated sampler for faster convergence.

Findings

Hessian-based acceleration achieves convergence rate of ~1/ε

Discretization schemes significantly influence convergence behavior

Quantitative comparison of Euler, exponential integrators, and midpoint methods

Abstract

Score-based diffusion models have emerged as powerful tools in generative modeling, yet their theoretical foundations remain underexplored. In this work, we focus on the Wasserstein convergence analysis of score-based diffusion models. Specifically, we investigate the impact of various discretization schemes, including Euler discretization, exponential integrators, and midpoint randomization methods. Our analysis provides a quantitative comparison of these discrete approximations, emphasizing their influence on convergence behavior. Furthermore, we explore scenarios where Hessian information is available and propose an accelerated sampler based on the local linearization method. We demonstrate that this Hessian-based approach achieves faster convergence rates of order $\widetilde{\mathcal{O}}\left(\frac{1}{\varepsilon}\right)$ significantly improving upon the standard rate $\widetilde{\mathcal{O}}\left(\frac{1}{\varepsilon^2}\right)$ of vanilla diffusion models, where $\varepsilon$ denotes the target accuracy.