Convergence guarantee for consistency models

TL;DR

This paper establishes the first convergence guarantees for Consistency Models, showing they can efficiently generate high-quality samples in one step under realistic assumptions, matching state-of-the-art results for score-based models.

Contribution

It provides the first rigorous convergence analysis for Consistency Models, demonstrating their effectiveness and scalability in one-step generative sampling.

Findings

Consistency Models can generate samples with small $W_2$ error in one step.

The convergence guarantees hold under realistic assumptions without strong data distribution constraints.

Multistep sampling can further reduce errors compared to single-step sampling.

Abstract

We provide the first convergence guarantees for the Consistency Models (CMs), a newly emerging type of one-step generative models that can generate comparable samples to those generated by Diffusion Models. Our main result is that, under the basic assumptions on score-matching errors, consistency errors and smoothness of the data distribution, CMs can efficiently sample from any realistic data distribution in one step with small $W_2$ error. Our results (1) hold for $L^2$-accurate score and consistency assumption (rather than $L^\infty$-accurate); (2) do note require strong assumptions on the data distribution such as log-Sobelev inequality; (3) scale polynomially in all parameters; and (4) match the state-of-the-art convergence guarantee for score-based generative models (SGMs). We also provide the result that the Multistep Consistency Sampling procedure can further reduce the error comparing to one step sampling, which support the original statement of "Consistency Models, Yang Song 2023". Our result further imply a TV error guarantee when take some Langevin-based modifications to the output distributions.