Operator splitting based diffusion samplers and improved convergence analysis

TL;DR

This paper introduces operator-splitting diffusion samplers that improve convergence analysis by providing sharper error bounds and demonstrating their effectiveness through numerical experiments.

Contribution

It develops a novel class of diffusion samplers using operator splitting and offers a refined non-asymptotic error bound analysis, surpassing existing results.

Findings

Sharper non-asymptotic total variation distance bounds established

Numerical experiments confirm quadratic error dependence on step size

Second-order samplers outperform previous methods in convergence analysis

Abstract

In this paper, we develop a class of samplers for the diffusion model using the operator-splitting technique. The linear drift term and the nonlinear score-driven drift of the probability flow ordinary differential equation are split and applied by flow maps alternatively. Moreover, we conduct detailed analyses for the second-order sampler, establishing a non-asymptotic total variation distance error bound of order $O(d/T^2+\sqrt{d}\varepsilon_{\mathrm{score}}+d\varepsilon_{\mathrm{Jac}})$, where $d$ is the data dimension; $T$ is the number of sampling steps; $\varepsilon_{\mathrm{score}}$ and $\varepsilon_{\mathrm{Jac}}$ measure the discrepancy between the actual score function and learned score function. Our bound is sharper than existing works, yielding bounds of $O(d^p/T^2)$ with some $p>1$ for specific second-order samplers. Numerical experiments on a two-dimensional synthetic dataset corroborate the established quadratic dependence on the step size $1/T$ in the error bound.