Score-of-Mixture Training: Training One-Step Generative Models Made Simple via Score Estimation of Mixture Distributions

TL;DR

Score-of-Mixture Training introduces a simple, stable framework for training one-step generative models by estimating scores of mixture distributions, achieving competitive results on CIFAR-10 and ImageNet 64x64.

Contribution

It presents a novel score estimation method for mixture distributions that simplifies training of one-step generative models and supports both from-scratch training and distillation.

Findings

SMT is simple to implement and requires minimal hyperparameter tuning.

SMT/SMD achieve competitive or superior performance compared to existing methods.

The approach is stable and effective on CIFAR-10 and ImageNet 64x64 datasets.

Abstract

We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen--Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64x64 show that SMT/SMD are competitive with and can even outperform existing methods.