VCT: Training Consistency Models with Variational Noise Coupling

TL;DR

VCT introduces a variational noise coupling framework for consistency training, reducing variance and improving image generation quality, achieving state-of-the-art results among non-distillation methods on CIFAR-10 and competitive performance on ImageNet.

Contribution

The paper presents a novel variational noise-data coupling scheme for consistency training, enhancing stability and performance across multiple image datasets.

Findings

Surpasses baselines in image generation quality.

Achieves state-of-the-art FID on CIFAR-10 among non-distillation CT methods.

Matches SOTA performance on ImageNet 64x64 with only two sampling steps.

Abstract

Consistency Training (CT) has recently emerged as a strong alternative to diffusion models for image generation. However, non-distillation CT often suffers from high variance and instability, motivating ongoing research into its training dynamics. We propose Variational Consistency Training (VCT), a flexible and effective framework compatible with various forward kernels, including those in flow matching. Its key innovation is a learned noise-data coupling scheme inspired by Variational Autoencoders, where a data-dependent encoder models noise emission. This enables VCT to adaptively learn noise-todata pairings, reducing training variance relative to the fixed, unsorted pairings in classical CT. Experiments on multiple image datasets demonstrate significant improvements: our method surpasses baselines, achieves state-of-the-art FID among non-distillation CT approaches on CIFAR-10, and matches SoTA performance on ImageNet 64 x 64 with only two sampling steps. Code is available at https://github.com/sony/vct.