Provable Statistical Rates for Consistency Diffusion Models

TL;DR

This paper develops the first statistical theory for consistency diffusion models, providing estimation rates and insights into their training methods, which enhance sampling speed without sacrificing quality.

Contribution

It introduces a formal statistical framework for consistency models, analyzing their training as a distribution discrepancy minimization and deriving Wasserstein-based estimation rates.

Findings

Establishes the first statistical estimation rates for consistency models.

Shows that consistency models can be trained via distillation and isolation methods.

Demonstrates that these models match the statistical performance of vanilla diffusion models.

Abstract

Diffusion models have revolutionized various application domains, including computer vision and audio generation. Despite the state-of-the-art performance, diffusion models are known for their slow sample generation due to the extensive number of steps involved. In response, consistency models have been developed to merge multiple steps in the sampling process, thereby significantly boosting the speed of sample generation without compromising quality. This paper contributes towards the first statistical theory for consistency models, formulating their training as a distribution discrepancy minimization problem. Our analysis yields statistical estimation rates based on the Wasserstein distance for consistency models, matching those of vanilla diffusion models. Additionally, our results encompass the training of consistency models through both distillation and isolation methods, demystifying their underlying advantage.