Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions

TL;DR

This paper analyzes the convergence properties of consistency models with multistep sampling under mild data assumptions, demonstrating their effectiveness in approximating target distributions in various settings.

Contribution

It provides theoretical convergence guarantees for consistency models with multistep sampling under general data assumptions, extending understanding of their approximation capabilities.

Findings

Samples are close to the target distribution in Wasserstein distance for bounded or fast-decaying tails.

Additional smoothing steps improve total variation distance convergence.

Multistep sampling enhances sample quality in practical diffusion model scenarios.

Abstract

Diffusion models accomplish remarkable success in data generation tasks across various domains. However, the iterative sampling process is computationally expensive. Consistency models are proposed to learn consistency functions to map from noise to data directly, which allows one-step fast data generation and multistep sampling to improve sample quality. In this paper, we study the convergence of consistency models when the self-consistency property holds approximately under the training distribution. Our analysis requires only mild data assumption and applies to a family of forward processes. When the target data distribution has bounded support or has tails that decay sufficiently fast, we show that the samples generated by the consistency model are close to the target distribution in Wasserstein distance; when the target distribution satisfies some smoothness assumption, we show that with an additional perturbation step for smoothing, the generated samples are close to the target distribution in total variation distance. We provide two case studies with commonly chosen forward processes to demonstrate the benefit of multistep sampling.